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S,  This  is  the  first  detailed  report  on  Contract  AP  53(6l6)-695S, 
^iceraed  with  th€!  fundaotantal  investigation  of  adaptive  control 
sya^niB. 

A  generaLL  survey  of  modem  analytical  methods  of  control  thoor:/ 
is  presented,  with  emphasis  on  special  topics  relevant  to  ad.aptive 
system  problems*  In  addition,  it  is  shown  hov  these  njcthods  am  Im¬ 
plemented  by  means  of  dlgitail  ccnputero.  A  oet  of  nev  lantrlx  nub- 
r'outines  is  described  in  detail. 


To  render  this  report  &a  nearly  self-contained  an  vas  considered 
ft'aslble,  a  conprehensive  appendix  nks  been  Included,  Thlu  apper-dl  '' 
is  referred  to  in  the  body  of  the  report  as  [Kalman,  1961  Cj, 
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IM  ^OHICTIOK 


ThlB  is  the  first  detailed  report  on  Contract  AF  55(6l6)-^52. 

This  contract  la  jiea’t  of  a  continuing  effort  in  the  broad  study  of 
cooplex  syateme  porobleais.  Accordingly,  it  was  thought  that  an  encyclopaedic 
presentation  of  all  roaults  obtained  to  date  vae  not  particularly  desirable; 
instead  we  liave  concentrated  or  giving*  a .  cartiful  account  of  the  conceptual, 
and  mathematical  foundations  of  the  reseajrch  problejns  Involved.  It  is  hoped, 
therefore,  that  the  report  will  serve  as  an  introduction  to  papers  on  modem 
system  theory  which  axe  now  appearing  In  the  literature  with  Increaslrig  fre¬ 
quency. 

The  report  consists  of  two  main  parts. 

Chapters  1-6  give  the  technical  and  mathematical,  background  of  our  pre¬ 
sent  approach  to  system  theory,  with  pairtlculor  attention  to  the  adaptive  pro¬ 
blem.  This  pext  is  primarily  concerned  with  a  clear  exposition  of  the  I'unda- 
mentol  ideas,  with  nunerous  illustrative  examples.  No  attempt  has  been  mode  to 
state  all  mathematical  facts  with  absolute  precision,  and  in  particular  most 
proofs  are  omitted.  Further  details,  which  are  often  very  involved  and  technioa.1, 
may  be  found  in  the  references. 

Chapters  7-11  are  concerned,  with  the  motivation  ond  description  of  digi¬ 
tal  computer  techniques  used  for  automatic  synthesis  of  optimal  systems.  These 
methods  are  a  fairly  radical  departure  from  cui-rent  engineering  practices  in 
the  systems  field.  They  Eire,  therefort;,  explained  in  considerable  detail.  It 
Is  expected  that  this  material  will  eventually  be  Incorpcratcd  into  a  "handbook;" 
of  Instructions  for  the  everyday  usage  of  the  autcmatic  optimization  progj-am 
which  we  axe  now  developing.  Besides  a  complete  description  of  the  aubroutlnes, 
a  number  of  check  programs  and  solutions  are  given  which  should  facilitate  use 
of  the  program  by  others. 


Manuscript  released  by  author  51  March  1961  as  an  A3D  TechnlceJ,  Report 


It  18  difficult  to  giYe  a  fair  deRcription  of  the  sdvriotagea  ve  1 1  nr- 
Bhortccoings  of  tbe  aethode  used  in  thiB  repoi’t  or  of  the  original  contrlbii- 
tionJB  which  are  iirrolrod.  Two  i«uln  aapectc  oiiould  be  omphMl/edj,  hov^-.  'i  r. 

fl)  A  coBCprcheMlve  analytical  theory  and  method  of  linear  Byot«r,  optiral/.a 
tion  has  been  Jr-reiopcd  (with  partial  support  of  thin  contract)  and  iB  ct)r>-*  nl 'y 
nearing  completion#  The  nost  impoirtant  featxurco  of  this  method  arc: 

(a)  It  Ib  new  and  quite  different  from  convent lonal.  mettKxi?.. 

(b)  It  lo  applicable  to  linear  syntemn  of  any  dcgi'ee  of  ccmplexlty; 
in  other  vordu,  no  modlfl cations  are  needed  to  treat  inultj -inhit 

or  multi-output  systems# 

(c)  It  is  applicable  in  principle  without  any  modification  (but 
possibly  at  great  cost  of  ccsssputatlon)  to  linear  r?yotemn  vi  tc 
varying  coefficients. 

(d)  It  provides  a  unified  treatnient  of  control  and  fl  Iterir-J^r,  probh  , 
coittbiuatilonB  of  the  two,  etc. 

(e)  It  provides  a  canonical  block,  diagram  for  the  opttisBLl  cyatem 
which  can  serve  as  the  starting  point  of  engineering  design. 

(f)  It  is  well  suited  to  high-speed  digital  comiutation. 

The  last  property  of  the  new  method  of  (r>(?met,iincr>  c;alj.ed  the 

"state- transition"  method)  glvws  rise  to  the  second,  important  contrlbut Ion  oi 
the  report; 

(p)  A  cfwipmbeTiHive  systern  of  n'JUDeric&l  co!!;j>utation'<  is  being  devcloyr'd 
to  Implement  the  theory.  The  computer  programs  will  be  'hiutomated"  to  a  very 
l  arge  extent  and  eventusJJLy  it  is  hoped  that  they  wxxi.  ue  ’ ^ ^ 

without  detailed  theoretical  training.  Specifically,  the  following  hui;  bec’^’ 
accoaplinhed  so  fesr: 
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(a)  Matrix  Giitroutiaes  have  been  developed  which  :'epcreaent  a  new 
approach  to  the  usual  computatioiial  probleias  of  transient 
responsve^  Btability,  etc. 

(b)  TThese  Bubi*outtne8  are  quite  sljnple  from  the  rjnthematica.1  point 
of  vie^;  and  allow  good  control  of  mnnericol  errors. 

(c)  Tb,a  method  of  c computations  le  "eigetwalue-free'*;  In  other  words, 
it  does  not  entail  the  solution  of  aigebredc  equations  of  hi^h 
degree  which  ia  charactcrietic  of  conventioruxl  techni 

result,  our  methods  con  be  eiccendcd  much  more  easily  to  loj  j'c 
scale  aystema  than  the  convent lonol  oneo. 

For  ease  of  croca- referencing  with  the  other  volumes  of  this  report  -  vhirh 
ore  to  be  laaued  later  -  each  chapter  ia  written  in  no  oelr-coo  ^aineo  a  v/ay  fir. 
pobBlble.  Fquationa  eird  figurea  ore  numbered  oejvirntely  vitlfLn  each  chapter. 

I' 

References  are  made  by  author  and  veer  of  publication;  cy.ch  chapte.T’  concsilni'  its 
own  liet  of  references,  even  when  thJ.a  entails  nosae  duplication;  duplicate  j-c- 
ferences  ootairrlng  in  different  cbapters  are  designated  in  a  lonslotent  rashior. 

Since  f^tpj.lg  of  the  watbejnatical  arguments  used  in  this  jxfport  are  not  yet 
readily  available,  a  recent  paper  by  B.  E,  Kalman,  "Bew  Methods  arnl  Kcsullr>  in 
Linear  System  Theory”  is  included  as  em  Appendix.  This  papf^r  contains  a  very 
extensive  diacussion  of  the  theoretical  aspects  of  the  optimal  filtering  problem. 
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Chapter  1. 


CONCEPTUAL  BACKGROUND 


--  •  Introduction . 

The  fundamental  mathematical  problem  in  the  design  of  a  control  ayHtem  is  the 
specification  of  the  control  lav. 

We  axe  given  a  dynamical  system  to  be  controlled^  called  the  contix)!  ob.}(ict. 
Examples:  (l)  an  airplane,  (ll)  a  satellite,  (ill)  a  chemical  plant.  Information 
abovit  the  physical  behavior  of  the  control  object  le  conveyed  by  i.ieans  of  certain 
physical  laeaaurenents  z(t).  These  measureinents  may  be  (i)  altitude,  Mach  number, 
pitch  angle,  etc.,  of  an  airplane;  (ii)  distance  of  a  natelilte  from  the  moon; 

(ill)  composition  of  a  chemical  formed  in  a  reactor.  The  behavior  of  the  dynamical 
liyatem  may  be  affected  by  changing  certain  physical  p»irameter«  u(t),  called  con¬ 
trol  variables.  Control  may  be  exerted  through  (l)  allv^ron  deflection  or  engine 
throttle  in  an  airplane;  (ll)  Jets  or  flywheels  inside  a  satellite;  (ill)  heat  oi' 
cateOysts  Influencing  the  rate  of  a  cliemicRl  reaction.  The  control  lav  is  a  pre- 
ecrlptlon  for  determining;  the  instantaneous  values  of  the  control  variables  u(t) 
on  the  basis  of  present  and  past  roeasuremertts  of  z(t).  The  control  lav  may  also 
depend  on  certain  other  parameters  specifying  the  desired  behavior  of  the  dynamica'i 
system  under  contix)!. 

The  problem  of  determining  a  control  lav  can  be  easily  stated  In  conceptual 
terms,  but  the  precise  mathematical  formulation  Is  not  a  simple  nuitter.  Caix'lTil 
assumptions  ■must  be  made  conce:  olng  the  mathematical  model  wliich  is  to  rcpresfuit 
the  control  object,  and  one  must  specify  In  what  sense  the  control  law  la  to  be  op¬ 
timal.  Without  a  clearly  defined  model  and  a  clearly  understood  criterion  of  opti- 
aality,  sophisticated  mathematical  technlqxreo  ai-e  ujioalled  for,  perhaps  even  deirl- 
mental..  In  simple  ca'’‘'s,  tjicl-and-error  experimentation  will  lead  to  a  system  de¬ 
sign  which  will  be  intuitively  satisfactory  and  probably  nearly  optimal,  in  complex 

In  more  conventional,  tenps ,  zCt)  lo  called  the  output  and  u(t)  to  the  Input. 
This  usage  Is  r-atber  axoblguous  and  will  be  avoided  in  the  sequel 


V 


oases,  this  procedure  becomes  icefflclent  and  eometJjaeB  iiipos Bible.  One  must  rely 
on  mathematicaj,  reasoning,  a».d  t'nls  reqvdrcs  greater  preclnios  of  problem  fo:nau2a- 
tlon.  In  the  physical  sclances,  the  dangers  caused  by  sloppy  application  of  raathe- 
BifitlcB  can  be  cheekeu  by  physical  Intuition.  In  the  control  eystems  field  -  >rhlch 
deale  with  aan-made  objects  rather  thaji  observations  of  Natiire  -  yhysice.!  intuition 
is  not  always  a  reliable  g'ulde, 

Mariy  assumptions  Must  be  made  to  derive  rationally  a  particular  control 
law.  Critique  of  assumptions  is  especla-liy  important  when  one  starts  to  explore  the 
concept  of  an  adaptive  contTol  system.  Such  a  system  is  characterised  by  the  actual 
or  desired  Independence  of  its  control  law  from  overly  specific  assujsptionB  on  the 
nature  of  the  control  object.  To  put  it  crudely,  a  control  system  is  adaptive  if  it 
can  perform  well  (perhaps  after  a  short  start-Up  period)  without  detailed  prior 
knowledge  of  the  dy’namlcs  of  the  control  object.  An  luiaptlve  system  Ttnint  be,  there¬ 
fore,  cajpable  of  some  form  of  learning. 

Ideally,  an  adaptive  controller  should  do  Just  ao  good  a  Jot  In  conoroiiing  e 
supersonjLC  airplane  as  in  controlling  a  nylon  factoiy,  vlthont  being  ni)ec  Ificai  ly 
designed  for  either  Job.  Tlie  gap  between  such  desiderata  anrl  the  present  ntate  of 
technology  is  very  great  indeed.  Tills  ban  led  us  (and  other  r*5oearch  werkerc.  :n  tric 
control  systems  field)  to  re-exemlnc  the  bases  of  present  knowledge  in  nn  attc'i'.pt  to 
see  why  the  adaptive  control  problem  seems  so  exquisitely  difficult.  And, of  ccrarcc, 
there  Is  also  a  very  real  need  for  a  better  theory  in  order  to  cvaluai  <•  and.  '  ’ 
possible  -  understand  and  generalise  numeroun  Intuit lv»r  prcpocals  now  bcln^;  made 
practical  adaptive  systems. 

In  the  initial  phase  of  research  on  adaptive  systems  wc  have  assumed  that  the 
equations  of  motion  of  the  control  object  are  Known ,  and  have  been  co'-.ccmed  primar¬ 
ily  with  the  rigorous  mathematic'O  fornoilatlon  and  effective  numerical  solution  of 
the  control  problem. 

The  moot  interesting  problems  in  adaptive  control  arise  when  thl.'  asfmmpi i on  Is 
relaxed;  extensive  pr'paration  in  necessary,  howevee^  before  we  can  reach  that  st-v^^e. 
In  essence,  it  is  necessary  to  put  conventional  control  theory  In  a  clearer  and  moiv 
precise  form  -  a  process  which  will  be  seen  to  yield  important  results  and  sngg<u<L 
new  problems  even  in  conventional  control  theory. 

2.  Mathematical  Models  for  Dynamical  Systems. 

Fundamental  in  the  mathematical  description  of  a  dynamical  system  Is  the  concept 
of  state .  This  is  sJ  rnply  a  convenie:it  way  of  expressing  what  might  he  loosely  cai  .led 
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the  B^lnciple  of  Caueallty..  ft>T  clarity,  we  fonnsllze  this  Idea  as  follows. 

EKFIKITIQ^  OP  SSAIEE.  The  state  of  a  dytiamlcal  oystem  is  a  minjjmal  set  of  nvsn- 
hera  which,  specified  at  aBy  given  time,  eufflce  to  determine  oomplctely  the  futui-e 
evolution  of  the  Byatean,  provided  the  futxire  forces  actli-.g  on  the  syetem  arc  known. 


We  are  accuBtomed  to  represent  physical  dynamical  syetcioB  by  means  of  a  system 
of  n  differential  equatlonB  of  the  first  order.  The  state  of  the  Bystem  lo  then  n 
(finite-dimensional)  vector.  The  u  real  maabcrB  coristitutlrvic  the  state  vector  are 
the  n  initial  condition*}  needed  to  'oniijaely  specify  the  solution  of  the  differen¬ 
tial  equations.  An  example  of  this  sort  is  provided  by  particle  mechanics:  a  system 
of  K  particles  free  to  move  in  5-dlmensloK«LL  space  has  a  state  vector  of  6n  com¬ 
ponents,  made  up  of  the  JN  position  tmd  3N  velocity  coordinates.  Jn  some  cases, 
even  the  dimension  of  the  state  vector  may  be  Infinite,  os  in  partial  differential 
eqxiatlons.  In  other  cases,  the  number  of  states  may  be  finite,  an  In  modelo  for  di- 
glteO.  cOBjputers. 

By  the  eqxiatlons  of  motion  of  a  djiMunlcal  systen:  we  mean  a  i-ule  which  specifien 
how 'the  state  of  the  system  at  a  given  time  is  trensformed  into  other  ntaten  in  the 
future.  We  sbadl  also  refer  to  this  process  as  state  transition.  UnuaJ iy  the  equa¬ 
tions  of  motion  are  given  in  the  small;  that  Is  to  nay,  by  differential  equatloiK’. 
which  specify  the  InflnlteslmeLl  state  transition  corresponding  to  the  infiniteulmal 
change  t  -*  t  +  dt  In  the  time.  By  integrating  these  differential  equations  we  ob¬ 
tain  the  equations  of  motion  in  the  large;  that  in  to  nay,  wc  can  nptsc'if'y  the  state 
transitions  corresponding  to  arbitrary  changes  -+  in  the  tLne. 

As  is  ccnimon  practice,  we  shall  'usuaJ.ly  assume  that  the  equations  of  motion  a.i-e 
linear  differential  equations.  Without  sane  form  of  linearity,  explicit  mathematical 
treatment  of  the  equations  of  motion  is  seldom  possible.  We  emnhoBl^.?,  however,  that 
the  conceptual  framework  presented  here  remains  valid  also  in  the  nonllncai-  cose.  In 


?act,  vi+h  the  r-r^Dcnt  forEulatior.  of  the  dynnnical  prcblc: 


linear  to  the  nonlinear  is  quite  natural  -  which  is  not  the  case  with  other  methods 

(lap  li-v,-.;  transfo..—,  ..i-e\,u.!ncy  dosnaiu  methov...,  eL^, ,  ) 

A  sufficiently  general  mathematical  model  for  linear  dynsmlcni  syotemr.  Is  pro¬ 
vided  by  the  vector  equations; 


(2.1) 


dx/dt  *  F(t)x  +  r.(t)u(t)  +  J(t)w(t), 
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where 


(2.2) 

(2.5) 


y(t)  =  H(t)x(t), 
z(t)  »  u(t)  v(t), 


X  iB  an  n~vector,  the  state  of  the  eystem; 
y  is  a  p>vector,  the  output  of  the  system; 
z  Is  a  p-vector,  ihs  observed  output  of  the  system; 
u  Is  an  m-vector,  the  control  of  the  system; 

*  V  is  a  p'-vector,  repre sent Ihg  the  noise  In  the  measurement  of  y; 
w  Is  a  q-vector,  the  random  dlstxirbances  acting  on  the  system; 


We  assufflc  that  F,  G,  H,  J,  which  are  arbitrary  rectangular  matrices,  depend  con¬ 
tinuously  on  t. 


In  a  purely  schematic  way,  these  definitions  may  be  visualized  with  the  aid  of 
Figure,!. 

The  set  of  equations  (2.1-3)  Includes  most  of  the  situations  commonly  encountered 
In  engineering  practice  (see  numerous  examples  of  this  in  the  sequel).  A  similar  set 
of  equations  may  be  obtained  also  In  the  sampled -data  case.  Certain  compllcntionn  nay 
arise,  however,  if  contlnxiovis  and.  pulsed  elements  occur  In  the  same  system.  The  setting 
up  of  equations  then  requires  rather  complicated  "bookkeeping”,  for  the  details  of 
which  the  reader  may  consiilt  [Kalman  euui  Bertram,  1959]. 

It  will  always  be  assumed  that  v(t)  and  w(t)  are  gausslan  whlte-noioc  pro¬ 
cesses,  i.e. ,  their  values  occuiTlng  at  dlffei^nt  inotauts  of  time  arc  independent  gausnlan 
random  vectors.  This  can  be  done  with  virtually  no  loss  of  generality.  Vre-  can  repi-e- 
eent  the  general  gausslaii  random  process  as  the  output  of  a  linear  (possibly  In.’lnite- 
dlmenBloral)  dynamical  system  excited  by  white  noise.  (This  is  tbe 

Loeve -K9.rhunen  representation  theorem  [Loeve,  1961].)  It  is  physically  reasonable  to 
approximate  the  i-esultlnv  dynamical  uvatem  with  a  finite -dimensional  one.  (This  means 
that  the  power  spectra  of  v  and  vr  are  ajsumed  to  be  rational.)  The  state  vat Lubles 
associated  ir.  this  way  with  the  random  processes  v(t)  and  w(t)  can  be  combined  with 
the  state  variables  of  the  eystem  to  be  controlled.  In  other  words,  all  prcbloras  in 
which  the  accumptlonc  of  linearity  ijuid  gausolanixosa  hold  can  be  reduced  --  with  a  change 
of  variables  —  to  the  standard  forn’  (2.1-3). 
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3.  Adairtlve  Systems j  Leai^itng  States. 

So  far  the  concept  of  an  adaptlv<*  aystens  hao  been  dlacussed  in  rather  vp^e 
terma.  Certainly,  there  la  no  definition  at  present  of  an  adaptive  system  which 
meets  with  general  acceptance.  We  are,  therefore,  obliged  to  introduce  our  own, 
sootewhat  special,  definition.  This  is  done  os  Sv  matter  of  convenience;  we  do  not 
wish  to  claim  that  ours  la  the  unlj’’  reasonable  point  of  view  with  regard  to  "adapt¬ 
ation". 

DSFENITION  OF  AN  ADAPTIVE  CONTROL  SYSTEM.  A  control  ayatem  is  adaptive  if  it 
la  capable  of  changing  its  control  law  eis  a  result  of  measured  changes  of  the  con- 

f 

trol  object  and  its  environment  and  In  such  a  way  as  to  operate  at  all  times  in  an 
optimal  or  nearly  optimal  foahion. 

A  cyatem  with  a  fixed  control  law  may  operate  quite  adequate].y  In  a  changing 
environment.  Such  a  ayotem  may  he  more  prox>erXy  called  Inscnaitlve  or  invariant, 
rather  tham  adaptive.  The  word  "adaptive"  usually  carries  the  cc/nnototion  of  an  or¬ 
ganism  being  able  to  take  advantage  of  a  new  situation.  Hence  we  do  not  regard  a 
aysten  oa  adaptive  unless  it  la  aldo  optimal  in  acme  sense. 

The  operation  of  any  adaptive  control  s-stem  will  dei)end  on  two  groups  of  data: 

(l)  measured  (or  estimated)  values  of  the  state  variables  of  the  control  object,  which 
are  used  to  determine  the  instantaneous  values  of  the  control  variables;  (li)  measured 
for  estimated)  numbers  defining  the  equations  of  the  control  objecc  and  its  environ¬ 
ment,  which  are  used  to  determine  the  control  law.  The  first  group  of  numbers  den- 
crlbes  the  mooenttiry  behavior  of  the  control  object;  the  second  group  refers  to  "struc¬ 
tural"  characteristics.  For  instance,  the  pooltior,  velocity,  and  angular  momentum 
of  a  rigid  body  belong  to  the  first  group  of  data;  the  moss,  moaaent  of  Inertia,  and 
internal  constitution  of  the  body  belong  to  the  second  groiip. 

A  strict  distinction  between  these  two  concepts  is  not  always  possible,  of  cc/urae. 
In  specific  coses  this  is  unlikely  to  lead  to  confusion,  however,  since  we  are  accus¬ 
tomed  to  identifying  the  second  group  —  structural  chcracterlstlcn  ~  with  those  pro¬ 
perties  of  an  object  which  ore  unchangeable  or  change  slowly  in  time  relative  to 

the  first  group. 

We  aholl  call  the  first  group  of  data  the  dynamic  state,  and  introduce  a  special 
term  for  the  second  group  of  data. 
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DEFINITION  OF  USARNINO  STATE.  This  1b  the  "stete  of  knowledge''  -  exprressed  In 
nothe«atical  fora  -  concerning  all  equatlone,  etatiatical  d&ta^  performance  indlceo, 
etc.,  which  are  utlliKed  In  airriving  at  the  function  specifying  the  control  lav. 

In  other  worda,  the  learning  state  la  a  act  of  numhere  representing  all  the 
quintitatlve  Inforaatlon  which  an  engiaecr  would  use  Itj  rat legally  arriving  at  an 
optimal  control  law.  For  instance,  In  case  of  the  model  (2.1-5),  the  learning  state 
is  the  collection  of  numbers  making  up  the  matrices  F,  G,  H,  J,  statistical  infor¬ 
mation  ooncerniog  the  randem  processes  v(t)  and  w(t),  as  well  as  tht.  att<thc7naticBl 
specification  of  the  performance  index  which  is  to  be  ndnlmlxed  or  i-isaxl  rd  zed  by  oiv 
timal  control. 

As  time  parees,  the  "state  of  knowledge" 'is  likely  to  deteriorate,  unlees  fur¬ 
ther  Information  becomes  available  from  physical  measurements.  In  an  adaptive  eyo- 

tem,  these  measuremente  are  utilized  to  update  the  "state  of  knowledge".  The  vay  in 
which  the  measuremente  of  the  structural  characteristics  are  utilized  detc ralue u  the 
transition  law  of  the  leaumlng  states. 

In  short,  there  are  two  types  of  dynamic  processes  taking  place  in  an  adaptive 

control  system:  (l)  the  state  vaa*lBbles  of  the  control  object  ai-e  estimated  frem 

measurements  and  corresponding  control  action  is  taken  In  accoitUmce  with  the  control 
law  exl8tl»’g  at  a  given  mcmientj  (ii)  the  structural  cheirarterl sties  of  the  control 
object  and  Its  environment  are  monitored  by  another  mcaourenK.nt  process,  a..d  corres¬ 
ponding  adjustments  ai-e  made  in  the  optimal  control  law  from  time  to  time. 

The  concept  of  the  leau-nlng  state  Introduced  here  is  clearly  evident  also  in 
[Bellman  and  Kalaba,  1959].  A  schematic  plctx-re  of  an  adaptive  system  is  shown  in 
Figure  2.  We  shall  return  later  to  the  discussion  of  this  fij-u^re. 

k.  Examples  of  Adaptation. 

The  following  examples  give  what  we  feci  is  a  reasonable  interpretation  of  the 
notions  of  "structural  characteristics"  and  "learning  states". 

Consider  the  problein  of  manipulating  the  control  surf>acen  of  an  airplane  or 
missile  co  produce  latei’al  acceleration.  The  action  of  the  control  system  will  be 
influenced  In  the  main  by  the  following  typ  3  of  effects: 

(a)  Random  atmospheric  disturbances  of  various  sorta. 

(B)  Loss  of  liydraullc  fluid;  aging  of  vacuniD  tut)eB;  effects  of  tfuiperat ’re, 
moloturc,  radiation,  etc.  on  electronic  coinponentB. 


(C)  ]D«c.ro&slng  flULr  density  nt  higher  altitudes  vhich  (i)  decreases  the 

effectlTenees  of  the  control  eurfacesi  (li)  decreases  aerodynamic  dreg; 

(ill)  changes  the  statistical  properties  of  vlndgusts;  etc. 

The  classical  theory  of  a  control  system  [Truxal,  1955;  Nevton,  et  ai.,1957] 
considers  the  problen  of  randoia  disttorbanceB  as  part  of  OTjtlma].  Linear  design.  iTr 
have  einphasiied  this  point  by  including  randan  effects  in  the  model  (2.1-5).  Thus 
orJ^'  (B)  and  (c)  arc  to  be  regarded  as  ’’structural  changes”. 

One  of  the  uialn  xt-esons  for  the  xise  of  feedback  is  to  counteract  chs-ngo  or 
typo  (b).  Internal  feedback  is  used  to  render  control  equipment  largely  ifiscnsiti-vc 
to  changes  In  the  characteristics  of  electronic  and  other  componentn.  According 
to  our  vlcv,  emphasized  in  Section  5#  this  is  not  adaptation. 

Changes  of  type  (c)  are  usually  the  most  drastic;  they  affect  the  very  nature 
of  the  control  object.  In  ether  words,  chemges  of  tyjx;  (c)  are  not  clow  chnngi-c  ^  vhe 
environment  but  Substituting  ai;  entirely  new  environment. 

5*  Specific  Forms  of  the  Learning  State. 

The  process  of  acquiring  information  about  une  Btructi-xal  chamcler LbL l.cc  of 
the  control  object  and  its  environment  may  take  name  roue  foxirut:  clepcndint',  on  the 
nature  of  the  control  problem.  We  mention  briefly  come  of  the  problemc  which  have 
been  discussed  in  the  literature  [Aseltine,  et  Levin,  1958]. 

A  large  class  of  adaptive  ej'Btems  is  concerned  with  learning  the  equations  of 
motion  of  the  control  object.  In  oonie  casec,  this  leurnirg  proceiu;  rmy  l>e  quite 
simple  in  principle. 

For  instance,  we  might  be  able  to  express  the  lift  and  drag  coef'lM cicnt.c  of  an 
airplane  in  teims  of  the  mass,  Mach  number,  axid  altitude.  These  three  quantities  to¬ 
gether  wcrcJ.d  conetltute  the  learning  state;  if  they  can  be  directly  mcasufeil,  t  in- 
problem  of  adaptation  would  reduce  to  calculating  lift  and  drag  by  preuru]  Igneri 
formulRe  and  utilliting  the  number  sc  obtained  to  modify  the  control  Jaw.  Th1  c  t>qx‘ 
of  adaptation  can  be  only  moderately  effective  since  it  io  of  the  ojkui-  loop  tyyx-;; 
no  effort  1b  made  to  revlne  the  original,  foimuiao  giving  lift  and  drag  .In  tciTTir.  of 
the  measured  quant J.tles.  More  effective  (closed- loop)  adaptation  could  bo  obiuiined 
by  fitting  a  model  of  the  equations  of  motion  of  the  airplane  to  the  piiy»ji''«L  quant  ! 
ties  measured  duilng  flight.  If  the  data  procer.aing  can  be  done  fauL  cnouv-uh,  r-ch 
a  model  would  obviate  dependence  on  prior  aerodyx’amic  data  In  mfilntalrlng  r-  r'.fo! r- 1 y 
optljnal  control  law  imder  a  wide  variety  of  flight  condttloTis. 
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Another  example  shewing  the  need  for  a  more  sophisticated  learning  procese  1b 
the  following.  The  bending  modes  of  a  ballistic  missile  change  with  the  expenditure 
of  fuel.  If  there  is  an  accurate  measurement  of  the  loss  of  mass,  one  c.ouJd.  in 
principle,  compi;te  the  shift  in  bending  modes,  in  practice,  measuring  the  Iogo  of 
mass  with  sufficient  accuracy  wcnild  be  very  difficult,  vind  one  would  rether  nttimp*. 
to  measure  the  j nstantaneoua  bending  modes.  The  latter  ore  to  be  regarded  then  as 
constituting  the  learning  state. 

Different  jiroblems  of  adaptation  arise  when  the  ,^roblem  l-n  ?■ 't  to  Jesjm  the 
equations  of  motion  of  the  control  object,  but  to  estimate  the  landom  charactcrl8i;lc.o 
of  the  command  plgnals  which  the  control  system  is  to  follow.  (This  is  sometimes 
called  "input  adaptation".)  The  problem  might  be,  for  instance,  to  estimate  the  cor¬ 
relation  functions  which  are  needed  for  the  design  of  an  optima],  Wiener  filter.  The 
predictability  of  a  random  process  depends  on  being  able  to  represent  it  by  a  dynami¬ 
cal  model,  so  that  seme  "equation  of  motion"  of  the  random  prc<'eBr)  wovjid  constitute 
the  leea*nlng  state.  Thus  this  problem  is  quite  similar  to  the  preceding  one,  t,h(n.*gh 
there  may  be  many  vetrlations  in  the  details, 

^Finally,  let  us  mention  the  so-called  performance- ci’ I ter.i on  sensing,  or  ext.r>'mum 
adaptive  systems.  Oome  overall  performance  index  is  measured  experimentally  and  ouc 
attempt. s  to  find  a  control  lav  (by  trial  and  error)  which  optimizes  this  performance- 
index.  The  learning  states  ore  nere  parameters  describing  the  control  law,  and  the 
learning  process  consists  of  the  tria.l-and-error  adjustment  of  these  parameters:. 

Of  course,  the  division  of  an  adaptive  controller  into  the  two  sets  of  statc-u 
is  quite  arbitreoy.  It  la  difficult  to  conceive  of  a  physical  experiiJK--nt  whirl!  would 
always  distinguish  between  the  two  types  of  states.  The  division  is  made  as  e  imattcr 
of  convenience  in  attempting  to  give  a  workable  definition  of  adaptation,  and  Is 
strongly  motivated  by  scientific  tradition.  We  ui-e  used  to  representing  nbystceil  dyna- 
mical  systems  with  linear  or  ouaai-llnceLr  models. 

A  Dingle  representation  is  not  likely  to  fit  accurately  many  sltuntloiin  w.t  the 
same  time.  We  must  have  therefore  a  capability  of  changing  the  parameters  of  the  re ■ 
prec'  '"^'on,  l.e.,  the  equations  of  motion.  A  particular  learning  state  corresiwndB 
to  a  particular  equation  of  motion;  the  transition  in  the  learning  states  corresponds 
to  changing  estimates  of  the  instantaneoufi  equations  of  motion  of  the  control  object. 

If  the  learning  states  can  be  changed  rapidly  and  accurately,  the  adaptive  controller 
will  be  able  to  handle  nonlinear  control  objects;  the  learning  states  will  rei>resent 
the  best  Imitantaneous  linear  approximation  to  the  nonlintar  system. 
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SCH£M/\TlC  REPRESE^f^ATIONOFA  DYNAMICAL  SYSTEM 
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FIG.  2  SCHEMATIC  REPRESENTATION  OF  ADAPTIVE  CONTROL  SYSTTM 


Chapter  2 


THE  NOISB- FREDS  KSOIHATOR  ITOEi;^ 

1.  Aes^BSptioKg  aad  y-j^atloa. 

For  vectoivinatrix  notation,  sec  [ICRlwan-Bertra*,  i960]  or  the  Appendlt, 

It  will  be  aBsvtsaBd  that  a  axifflciently  oCiCurate  ac*€a  of  the  dynanlcal  systew 
to  be  controlled  is  provided  by  the  linear  equatloiuj  dlBcussed  in  Chapter  ] ,  Section 
2.  In  fiddltlon,  it  viU  be  assumed  thrr::gho«t  this  chapter  that  noise  effects  are 
absent;  in  other  words,  v(t)  and  w(t)  are  identlcoLly  B«ro,  Thus  vc  nhsU  be 
concerned  with  the  vector  equations 


(1.1) 

dx/dt  ■  F(t)x  +  Q(t)u(t) 

(1.2) 

y(t)  -  H(t)x(t). 

It  is  convenient  to  rlsviallzc  this  systen  by  ueeuaw  of  a  vector  block  (Uagraw 
shown,  in  Figure  lA.  This  dlagraa  is  to  be  interpreted  Jnst  ar.  cn  ordirAry  block  dln- 
graa,  with  two  differences;  (l)  the  fat  lines  used  to  denote  the  tilgjial  flow  nei-vc 
as  a  reminder  that  we  are  dealing  with  vector  rat2ier  than  ocali’.r  variables;  (ii) 
txie  boxes  denote  linear  transformations  on  the  signals  rather  thxui  Kulltplication 
by  .icalars. 

In  concrete  terns  the  block  diagram  In  FLgiAre  lA  is  to  be  interpreted  as  follows. 
The  box  l/s  represents  a  s'*t  of  n  integrators.  The  output  of  the  J-th  irtetpvs- 
tor  is  fed  back  with  the  coefficient  ^n-nut  of  ti>*  j  -th  Integr^totr. 

The  J~th  control  variable  forward  with  coefficient 

input  or  the  1-th  integrator.  Finally,  the  1-th  output  y^(t)  is  a  linear  ec»- 
hination  of  the  outputs  of  all  the  integrators,  where  ihe  output  of  the  ^-th  integra¬ 
tor  appears  wl.th  coefficient  h^j(t).  See  Ifgurc  IB. 

We  Bhttll  assume  txist  iKt),  a(t),  H(t),  and  u(t)  are  piocewloo  contlnncus 

1:^ 


functions  of  time.  Then  given  a  fixed  control  u(t),  (l.l)  vill  hav«-  a  unioufr 

solution.  Aside  fr«xo  the  time,  this  solution  vill  depend  also  on  (i)  the  initial 
state  x^;  (li)  the  initial  time  t^;  (ill)  the  control  u(t).  It  io  often  conven¬ 
ient  to  exiilblt  this  dependence  explicitly;  we  shall  therefore  write  f.  solution  of 
(l.l)  in  the  fonn 

m  (t:  X  ,  t  ). 

'  O''  o' 

This  .lotatlon  implies  that 

't'u'  o'  o'  o'  f  o' 

and 


(l.U) 


dt 


x^,  t^)  +  G(t)u(t). 


The  hast  equation  is  the  definition  of  the  oolutlou  of  a  differential,  equatloti.  The 
equality  needs  to  hold  almont  everywhere  with  respect  to  t;  jmoro  prc-ciue.iy^  (i.*0 
may  fall  at  points  of  dJ.3Contlnuity  of  F(t),  G(t),  or  u(t). 

Instead  of  the  cumbersome  term  "solution",  we  shall  usually  speak  of 
as  the  motion  of  (l.l)  passing  through  the  point  x^  at  the  time  unrtei-  the  in 

fluence  of  some  fixed  control  u(t). 

It  is  well  known  in  the  theory  of  differential  equations  [CoddLng^'On  and  l/^vlnso-n 
19[35j  Kalman  and  Dei”tram,  I96O]  that  the  motions  of  (''.x)  can  be  expressed  explicitly 
by  means  of  the  formula 


(1.5) 


x(t) 


<P 


t  )x 
o  o 


J 

t 


0(t,  T  )i(t)u(  I  )d 


which  la  valid  for  any  x  ,  t,  t  (and  not  merely  for  t  k  t_). 

.  I.  '-A<i  Q*'  f  ty  Q 

The  matrix  ♦(!,  t^)  occurring  in  (I.5)  1^*  'the  transition  matrix  of  (l.l)  and 
la  uniquely  determined  by  the  following  renuirements  [Kali'ian  and  liertram,  19’-0]. 


(1.6) 


•(t,  t)  *  I  *  unit  matrix  for  ell  t. 


'.6 


d*(t,  t^)/dt  *  F(t)#(t,  t^)  for  all  t  and  t^. 


and 

(l.T) 

Fran  these  praporties  and  the  uniqueness  of  solutions  of  (l.l)  one  can  shoir  at  onnw 
that 

(1.8)  *"*(t,  t^)  «  •(t^,  t)  for  all  t, 

(1.9)  ^{ty  t2)«(t2,  tj^)  -  t^)  1‘or  all  t^,  t^. 

CONVESTION.  During  the  sequel  ve  shall  frequently  anlt  explicit  mention  of 
arguiaentB  (such  as  time)  If  they  are  obviously  Implied  by  the  context, 

2 .  Quadratic  Performance  Indices . 

We  now  define  the  regulator  problem.  Given  that  (l.l)  is  at  scree  arbitrary  state  x 
at  time  t,  we  arr*  to  select  a  control  u(t)  which  drlvec  the  output  o’'*  (l-i)  to 
tero. 

In  generJii,  there  will  he  many  control  functloruj  which  acconipllGh  this.  To 
assign  a  numerical  "alue  to  a  particular  control^  we  consider  the  following  function 
of  the  controile-  .ion,  uuxuU.ly  cedled  a  jKirfonnance  Index: 


(2.1)  2v(x,  t,  T;  u)  »  llcpjTj  X,  t)irg 

+  /'^[|!h(t)cp,.(t>  X,  t)||^^_,  +  iiu(a)l,l^,/^ Jot, 

t  " 

where  ve  use  the  8i)ecial  notation 


(2.2) 


|xl|^=  E 

l.h 


1,J=^1 


for  u  quadratic  form  In  x  whose  coefficients  constitute  the  ojirar-ctric  nonnegative 
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d»flxdte  afttrlx  Q.  The  acular  jjxH^  can  be  regarded  as  the  genentllzed  euclidean 
distance  froa  the  origin,  ^n  (2.1)  Q(t)  R(t)  are  posltlYc  definite  and  conticuoxia  Ir. 
T.  The  ter*  ii^^(Tj  x,  t)||g  In  (2.1)  le  the  cost  at  the  deviation  of  the. final 
state  of  the  dyneadcal  ejutea  froai  the  origin,  ineasvired  vlth  the  aid.  of  the  distance 
function  (ixllg  .  The  teraw  I!h(t)<j>^(tj  x, 

^(reaent  cuats  per  unit  tla«  of  tha  deviation  of  the  output  of  the  dyn«»lcal  system 
fre*  the  origin  and  the  cost  of  the  control  action  u(t)  .respectively. 

The  terainal  tlae  T  In  (2.1)  my  be  finite  or  Infinite.  In  the  latter  case 
spec?*-,  precautiorjs  ere  necessary,  as  the  Integral  (f.i)  must  be  defined  by  a  limit¬ 
ing  process,  letting  T  00. 

This  fonsulatlon  of  the  regulator  problem  can  be  reeulily  generalized  to  include 

f 

the  Bervoaechan^  iwi  problem.  In  that  case  ve  are  given  a  certain  desired  output 
y^(T)  vhlch  the  system  (l.l)  1^  to  follow  as  faithfully  oa  possible.  To  include 
this  requlreaent  In  the  definition  of  V,  we  simply  replace  the  first  term  In  the 
integrand  of  (2.1.)  by 


l|y‘^(‘r)  -  H(t)9^^(Tj  X,  . 

Further  discussion  of  this  problem  will  be  postponed  till  a  later  chapt.er. 

Eyldently  ^he  performance  Index  V  In  (2.1)  is  a  guadratlc  function  of  the  in¬ 
itial  state  X  for  any  fixed  u(t).  The  reason  for  this  assumption  is  that  It  leads 
to  a  linear  control  law. 

Finally,  It  should  be  noted  that  the  problem  becomes  meartlugleL-Lv  if  the  cost  of 
the  control  power  included  In  the  integrand  iu  (2.1),  for  then 

V  can  be  made  arbitrarily  small  by  uBlr.g  control  variables  of  1  ncrean It igly  large 
as^lltudcG. 


5*  Statement  of  the  Noise-Free  Optimal  Regulator  Problem. 

o..ven  that  the  motl....  of  (l.2)  pasoc-  .1.  .^ugh  the  point  x  at  time  t,  find  a 
control  u°(t)  wh.l.ch  minimizes  the  performance  index  V.  Tlie  minimum  vulu'-  of  V 
will  depend  only  on  x,  t,  and  T  and  can  be  denoted  by 

(^-l)  V°(x,  t,  T)  =>  min  v(x,  t,  T;  u) 

u 
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A  rigorouR  ti^atment  of  this  problem  may  be  fcnjnd  in  [Kalman,  1961  A~Bj.  We 
shall  now  sketch  the  main  features  of  this  theory,  casittlng  most  proofs  „ 

The  c^iraal  regulator  problem  is  a  sperJal  case  of  a  general  problem  In  theoreti¬ 
cal  physics  or  the  calculus  of  the  variations:  that  of  cinlmizlng  the  action,  which 
is  the  integral  of  the  lagrangian.  In  the  prc„jnt  cane, 


V°(x,  t,  T)  -  V“(x,  T,  T) 


is  the  action  and 


(5.2)  L(x,  u,  t)  [|lH(t)xi||^^j  +  Nt)|i^^j] 

T 

la  the  lagranelan. 

It  can  be  shcvn  further  [Kalman,  1961  A-B]  that  V°  satiofies  the  hamllton- 
Jacobl  partied  dJ  fferentlal  equation: 

(5.5)  ^  +<(x,  V°,  t)  -  0, 

If 

where  V°  is  the  gradient  of  with  resi)cct  to  x;  setting 

(5.4)  p  -  v°  , 
the  hamiltonion  3*0  Is  defined  by 

(5.5)  ^  (t:,  p,  t)  -  min  (l(x,  u,  t)  +  p'[F(t)x  +  o(t)u]} 


Vf V.J  r* V,  1  ^  ^  rj 


(5.6) 


u,  ?,  t) 


iiHVt)xii|(tj 


+  2p'F(t)x  -  ||0’(t)p[l^ 

R'\t) 


In  accordance  witVi  modern  usage,  adjectives  and  nouns  formed  from  names  of  tnatheBnatl- 
clonn  who  died  before  1900  are  not  capitalized. 

#» 


It'c  prime  uenotes  the  transpose  of  a  matrix  or  of  a  (column)  vector. 
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The  minlndzatlon  involved  in  (5.5)  Is  knovm  as  Pontryngin*ti  lalnimua  principle 
[Pontryagin,  155T;  Kalasan,  196I  B].  Tbla  principle  yielda  at  once  the  optlmnl  >  on- 
trol  lav  as  a  function  of  p,  and  t: 

(3.7)  u°(t)  «  -R‘\t)a'(t)p  , 

Hence  apeclficetion  of  the  optimal  control  law  reducea  to  finding  p  aa  a  function 
of  X  and  t,  in  other  wordc,  to  the  solution  of  the  hami It on- Jacobi  partial  differ 
ential  equation  (5.3) • 

Fortmils  (3.T)  is  valid  provided  R(t)  is  a  nonslngular  matrix.  More  generally^ 
an  optimal  control  law  exlste  if  the  right-hand  aide  of  (5. [3)  hna  a  roininpjLm  with  re¬ 
spect  to  u  —  for  which  the  nonsingularity  of  P  le  a  sufficient,  though  not 
necessary,  condition.  These  mathematical  facts  have  an  instructive  physical  ir.terpre 
tation,  ns  waus  pointed  out  at  the  end  of  Section  2. 

Solution  of  the  Kami It on- Jacobi  Equation. 

Equation  (3.3)  may  be  solved  by  assuming  that 

(U.l)  V°(x,  t,  T)  -  I  !lx||^t)  • 

There  is  no  lose  of  generality  in  assuming  that  P  is  symmetric.  Since 


making  use  of  the  symmetry  it  follovn  that 
(4.5)  P(t)  “  8. 

From  (3.4)  and  (4.1)  we  have. 


(4.4) 


p  -  P(t)x. 


Substituting  (4.1)  into  (5.5)  gives 
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(4.5)  I  x»  ^x+l  x'H’QEx  +  x'PPic-jx'PQR”^0*l>ic  «  0. 

This  iBUSt  hold  Identically  for  all  x;  hence  (noting  that  the  symnetrioal  part  of 
PF  iB  |(F’P  +  PF))  (4.5)  fllaplifies 

(4.6)  ~  «  F'P  +  PF  -  PQR"^G'P  +  H’QH, 

which  is  the  so-called  matrix  rlccatl  equation.  Hence  we  have  arrived  at  the  follow¬ 
ing  result;  *  : 

'  ( ^*7)  A  solution  of  the  handlton-.lacobi  partial  dlffer^^ntial  equation  corres¬ 
ponding  to  the  lagreinglan  (3«2)  the  hamlltonian  (3»3)  Ib  given  by  the  quadratic 
fora  (4.1),  with  time-varying  coerflcienlH  governed  by  the  matrix  r Iccati  equation 

(4.6)  .  This  solution  must  satisfy  the  boundary  condition  (4»g).  '.I'he  rlccatl  cq.ua- 

tlon  is  to  be  solved  BAiTKWARDS  in  time,  storting  with  p(t)  ^  S. 

#  ' 

It  follows  from  (5«7)  and  (4.4)  that 

(4.8)  u°(t)  .  -R-^t)G«(t)P(t)x(t), 

which  shows  that  the  optimal  control  law  is  linear.  Hence 

(4,9)  The  optimal  controller  is  a  llnoar  feedback  nyotem  In  which  all  ntgte 
variables  of  the  systen  met  be  known  at  all  times  to  effect  control. 


Tile  matrix 

(4.1C)  K(t)  *  R"^(t)G'(t)p(t) 

will  be  called  the  optimel  gain.  Figure  2  shews  the  vector  matrix  block  diagriun  of 
the  optimal  control  system. 

The  restriction  that  all  state  variab.  cs  be  known  at  all  times  will  be  removel 
in  a  later  chapter 
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5*  &P  th«  Theory  In  a  (?aa.e» 

m  or4.r  to  give  th.  foter  .  f«Uog  for  tho  Ootalls  of  the  theory,  . e  ehell 
glw  e  ccwleto  dl.coeeloa  of  the  flret-order  cue.  The  moOel  of  the  control  object 

is  t«lc«n 


(5.1)  ^  ■  ^11’=!  *  SuV  - 

f 

This  aeans  th*t  the  BStricea  defining  (l.l)  «re  given  by 


F  «  [^11^ 
G  *“ 

H  -  [\^]i 


all  these  loatrlces  are  asBumed  to  be  constant^  and  /  0,  /  0.  See  Fig. 


The  performance  Index  in  defined  by 


(5.2) 


2V°(x, 


■  / 


■  ••V  V; 

I' 


and  therefore 


q  - 

p  - 

a  -  [o^i3; 


all  these  matrices  are  also  assumed  to  be  constant.  Moreover,  >0, 


8 


U 


i  0. 

The  riccati  equation  (4.6) 


is  now 


0, 
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(5.3) 


Since  this  Is  a  first-order  nonlinear  differential  eq,uatlon^  it  is  easy  to  dls- 
CU88  its  behavior  in  qualitative  terss.  Equation  (5.3)  heiu  two  equilibrium  states, 
and  which  are  the  roots  of  the  quadratic  resulting  from  setting 

dp^j^/dt  =0: 


^^11 

dt 


>  0  if  p^^  <  <  p^^  , 


which  shows  that  p^^^^  is  a  stable  and  ^  Is  an  unstable  equilibrium  point  of 
(5.3)  -  00  , 

The  meamlng  of  letting  t  -♦  -00  in  (p*?)  is  rullowlng.  tjj,'  the  constancy 
of  q^^,  the  optimal  performance  Index  (?.2)  is  Independent  of  the 

origin  of  time: 


* 

(Remomber  that  dt  <0  in  thiccase  and  hence  the  irequallties  are  Just  opposite  ol 
of  the  usual  case  where  dt  >  0) . 
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t,  T)  -  0,  T  -  t). 

Hence  t  -♦  -oo  1b  equivalent  to  T  -»  oo. 

Then  the  llgdtlng  aolutlona  of  the  vaxltmcc  equation  as  t  -> -oo  correepond 
to  infinite  tenttlnel  time  T. 

Since  ^  clear  that  Oiily  the  eqiiilibrium  state  p^- 

is  of  iaterect.  We  shall  call  the  eteady-state  aolatioi:  of  the  riccatl  euaa- 

tion  (50) ■  Note  that  p, ,  Is  independent  of  s, ,  as  long  as  e,..  0.  Toe  correc- 

JLi.  ,LX  J.X 

ponding  Bteady- state  optimal  gain  is 


V  3  ^11  ..  ^11  ■ 

^11  ^11  r  g 

11 


2  ^li 


the  equations  of  motion  of  the  optimal  controller  are 


(r>.5) 


1 

dF*  “  ^^11  ■ 


eus  shovrn  In  PMgure  yB. 

2  2  - 

Recall  now  that  >  C.  Then  f ^  ^  <  0  and  w»*  iu*e  t  hat,  i  r; 

f-ita  lu  ih-^t  h'  i  i  i  j.-.  nj  j  i.  i  v  i.,  w,  u-.t'-*'!  '•• 

l.e.^  whether  the  uncontrolled  system  was  stable  or  unstable.  Hrifi  Is  not.  trivial 
because  the  mere  fact  that  a  system  Is  optimal  does  not  imply  that  it  iu  also 

stable  I 

Hot  only  is  the  optimal  system  stable,  but  an,y  degree  of  atabillty  can  be 
accontpllshed  by  suitable  choice  of  the  ratio 

If  we  can  write,  approxlaatclc,' 


2 


R*giirdlrg 


V°(xj)  .  i  PjjXf 

at*  H  T^.'»T>*ii'iov  fiiRctl  ori.  wc  li./-  +h«[t  Its  derivative  along  motion*  of  (5.I)  is  th« 
Integrand  in  (5.2): 


V°(Xi)  -  KiAi'*! 


Using  again  the  approximation  ^  ^  roll'-;wr>  that 


.  =  h 


11  “11 V  r,-  • 

xJ. 
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Hence 


Frcm  the  theory  cf  Lyapunov  [Kalston- Bertram^  19LX)^  p.  followc  that  tfie 

constant"  of  any  dynamic  system  (linear  or  nonlinear)  Is  bounded  by 


T 

o 


2  max 
^1 


V"(xj 


-V  U^l 


^^11^11 


That  In,  tV  V.  Regarding  V  as  a  measure  of  the  distance  of  the  state  from  thf 

«•  ^  ^'C 

origin,  this  leadn  to  i.he  estimate  V(t)  S  e  ^  V(0)  of  the  transient  response. 


Lattlng  p  "  euMMUflEe  these  insult b  as; 


(5.6) 


Sence  the  constant  of  the  optlml  system  can  be  inade  arbltrsj-lly  umall  by  let¬ 
ting  p  be  large,  hut  this  le  always  accceapllshed  by  increasing  the  gain  end 

hence  the  ait(plitudo  of  the  control  signal  u(t). 

The  question  now  arises  as  for  what  values  of  T-t  (5.5)  can  be  regarded  as 
hoving  practically  reached  Its  steady-state  value.  In  other  words,  on  what  depends 
the  time  constant  of  ^.5)7 
Let 
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Hence  the  nexlmuin  ttne  conatiuit  of  the  riccati  equation  Is 


(5.7) 


11  Pii 


2[g 


11  r 


11 


Evidently  the  more  stable  is  the  optiaal  control  eystem,  the  "faster”  does  the  solu¬ 
tion  of  the  jciccatl  equation  approach  its  limiting  value. 

To  Buaoaarize,  ve  have  found  that: 

t 

(l)  if  P3^i^(T)  *  0,  all  solutions  of  the  riccati  equation  tend  to  p^,  as 

t  -♦  -oo; 

(ll)  ^  is  the  solution  of  the  optimal  regulator  prohlem  when  T  *  oo; 

(ill)  if  ^  optimal  Bystom  la  always  stable; 


(iv)  by  making  the  ratio  ‘lii/-'‘]Ll  desired  degree  of  stability  can 

be  obtained; 

(v)  the  time  constemt  of  the  riccati  equation  is  directly  related  to  tue  time- 
constant  of  the  optimal  filter. 


The  main  aim  of  the  theory  of  the  optimal  regulator  problem  is  to  extend  the  re¬ 
sults  to  aystemo  of  higher  order  and  to  nystems  with  time- varying  coefficients.  This 
requires  fairly  coinplex  matrix  analyBls,  and  vlll  be  dJ-ocusaed  later.  For  further 
details,  consult  [Kalman,  1961  A-C]. 


6.  Existence  of  Solutions  of  the  Optimal  Regulator  Problem. 

The  main  result  here  is  expressed  by  the  following  oheor^,  which  is  proved  in 
[Kalman,  196I  A]. 
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(6.1) 


Mn  has  a  solution  for  every  finite 


T  -  t  If  the  matrix  R(t)  ard  the  cacktrix  (i(T)  are  positive  definite  for  J.1  r  in 
the  Interval  (t^  T),  vfalle  S  is  nonnegative  definite. 

In  order  to  iinlerstand  this  result,  a  simple  counter-example  vill  be  considered 
In  detail.  Define  a  performance  Index  by 


(6.2) 


2V°(x^,  t,  ’f)  r.  tnax(s^^x^(T)  +  /  [y^(T)  -  ^(f)]  dt ) 


•;*hile 


(6.5) 


dx^(t) 


coCi(t)  +  u^(t)  (a  =  real), 


In  other  vords, 


F  -  (a),  G  =  (1),  =  1,  h  1,  r  -  -1 


The  contjepondlng  equation  has  a  solution 


2V®(x^,  t,  T)  =  P3^i(t)xJ, 


vhere 


p^d)  -  =11 


f  ^  ♦  > 


dp,,(t)  2 

—  -  1  +  aTa?j^^(t)  +  P  (t), 


Integrating  (6.4)  by  separation  of  vexlableo,  ve  get,  setting  =  0, 
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r 


Pii(t) 


(T-t) 

1  -  -£^-5  (T-t) 

^1-0 


if  aP  <  1, 


(6.5) 


» 


Pii(t) 


taA  vc^-1  (T-t) 

^  Va^ 


“  i  +(T-t) 


V 


Pii(t) 


T-t 

i  -  fT-t) 


If 


>  1, 


If  a  »  -  1; 


if  a  *  +  1. 


Fx*oin  (6.5)  it  cun  be  observed  that  If,  a  >  -  1  the  solution  ^ 

finite  escape  tine  and  the  maximization  problem  is  MEARINCJLKSS  for  T  -  t  >  t^ 
where 


(6.6) 


For  a  <  -  1  a  solution  exists  in  the  steady  state,  that  is,  T  ->  +  00  If 
8^^  *  0.  However,  if  /  0  and  is  a  sufficiently  large  positive  number,  even 

when  a  -  1  therti  is  no  steady-state  solution.  This  phencmenon  is  shovm  clearly 
by  the  state-space  ( 1- dimens iunal)  plot  of  ^he  differential  equation  (6.U).  See 
Figure  !?,  where  the  arrows  indicate  the  direction  of  motion  as  t  -»  -00.  Indeed  if 
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*11  ^  -  1,  then  no  steady  state  solution  exists.  These  cases  illustrate 

the  |>rohleeM  vhlch  one  isay  encounter  hy  negleotlng  existence  questions. 

Bse  classical  approach  via  the  euler  equations  would  not  roveal  the  fact  tliat 
the  optimisation  problem  hecoaes  weanlnglCBS  for  large  T  -  t;  the  ctiler  eq^iatious 
in  the  present  case  always  have  a  unique  solution. 

• T*  BxlBtence  of  the  Solution  of  the  Steady-State  Oytliwil  Regulatur  Probleg. 

The  optimal  regulator  problem  makes  sense  In  the  steady-state  (T  =  cO  only 
if  the  limit  * 

(7.1)  '  V(x^  t,  00;  u)  t.  lim  V(x,  t,  T;  u) 

T  -♦  00 

exists  and  is  finite  for  aooo  control  function  u)t)  defined  for  t  e(t,  00). 

In  order  to  investigate  this  situation,  we  introduce  a  new  concept: 

«  ' 

nSFTNinOH  OF  COMPLEIS!  CONTROLLABILITY.  A  system  (l.l)  is  said  to  be  completely 
controllable  if  at  any  initial  tiaae  t  any  initial  state  x  can  be  l^.ken  to  the 
origin  in  a  finite  length  of  time  by  the  application  of  a  suitable  control  function. 


The  abstract  definition  of  complete  controllability  is  equivalent  to  the  follow¬ 
ing  concrete  condition  (for  proof,  see  iKalman,  Ho,  Narendra,  1962)): 


(7.2)  -rHEOREM.  A  system  (l.l)  is  completely  controllable  if  and  only  if  tlie 

matrly 


(7.3) 


T 


w(t,  T)  =  J  0(t,  T)G(.): 


\m.  t  f' 4.  ^  \ 

<  •  V  /  '  /  - 


is  positive  definite  for  some  T  >  t. 


The  Si.'rttriJi  W  has  the  following  Interpretation*  The  alnjjmjm  energ/  required 
to  transfer  the  state  x  at  time  t  to  the  origin  at  tJjne  T 

-  l|xf  . 

t  W-"(t,  T) 

The  computation  of  the  matrix  W  is  most  easily  performed  by  obaer'/ln/j:  that  it, 

too,  is  governed  by  the  rlccatl  equation.  To  see  this/  we  note  two  facte:  (l)  if 

a  matrix  P  is  governed  by  a  liccati  equation,  then  its  Invcroe  P“^  (if  it  exieta) 

la  also  governed  by  a  rlccatl  equation,  (ii)  By  the  remai-k  in  the  precodine;  para- 

graph,  llxjl  .  is  the  performance  index  for  a  oi>ecial  optimization  problem: 

W“  (t,  T)  *  ; 

take  X  at  time  t  to  the  origin  at  time  T,  minimizing  along  the  motion  the  con¬ 
trol  energy. 

Differentiating  (7*5)  with  reapect  to  t,  ualng  (i.7-9),  we  find 
(7.^)  dW/dt  =  F(t)W  +  OT”(t)  -  G(t)R"^(t)G'(t) 

I  ' 

which  is  a  special  ceise  of  the  rlccatl  equation  (*+.6).  ’^n  practical  caoco,  W  In 

usually  computed  by  means  of  this  equation  rather  than  by  numerical  integration  of 
(7.3). 

If  the  system  (l.l)  is  constant  (or  stationaj-y )  that  in  to  nay,  if  F,  and 
G  are  constants,  then  complete  controllability  can  be  checked  more  simply  (for  proof, 
see  fKeuman,  Ho,  Harendra,  1961]  2): 

(7.5)  theorem.  For  a  constant  system  (l,l)  a  necessm'y  and  pufflclent  condition 
for  courpl-ctc  controllability  la 

(7.6)  rank  (G,  FG,  f’^’’^g1  *  n. 

The  condition  of  complete  controllability  is  not  eor  the  existence  of 

the  limit  (7.1).  But  if  a  system  is  not  completely  cout.  rollublr,  its  state  variables 
can  be  decentposed  Ir.io  tvo  groups,  one  of  which  Is  completely  unaffected  by  control. 
See  Figure  6/.  If  the  paid,  of  the  system  which  is  not  coupled  to  u  is  asyiusptotj.cally 
stable,  then  the  limit  (7.!)  exists j  but  If  this  part  is  unstable,  the  limit  will  not 
exist. 


It  can  be  ehown  th&t  if  a  liaear  coaetaat  nyetcm  is  deucribed  by  a  transfer 
function,  then  It  Is  nl^rayis  ccsBpletely  controllable.  This  is  due  to  the  fact  that 
in  writing  down  the  transfer  function  tents  which  account  for  lack  of  complete  con- 
trc-llsblllty  cancel  out  of  the  nwesrator  and  dencalnator. 

Because  a  slngle-input/slngle-output  control  object  described  by  a  transfer 
function  is  always  coaopletely  controllable,  the  importance  of  controllability  wns 
unnoticed  for  a  long  time  In  the  lltaraturo  of  control  engineering.  In  simple  cases, 
lack  of  controllability  is  easily  detected  and  eliminated  by  physical  considerations. 
On  the  other  hand,  in  conplicated  cases  when  the  ©q,uaT  i.or>r  Oi:  motion  tire  written  in 
the  normal  form  (l.l)  and  there  are  several  inputs  and  outputs,  controllability  is 
not  obvi  ouB  and  efficient  mathematical  means  iinust  be  Cevlsed  to  test  this  property 
of  the  system.  This  is  the  price  one  has  to  pay  fer  a  more  general  theory. 

If  we  do  not  have  cojsplete  controllability,  the  limit  (t.I)  will  not  exist  in 
general.  This  is  easiJy^  seen  hy  the  following  example 


dx,/dt  =  f^^x^  +  u^(t)  , 
dx^dt  =  x^, 
dxy'dt  -  -Xg, 


Q  »  [1]  . 

R  -  [il, 

8  « 


0  0  0 
0  10 
0  0  0 


Then  V  will  always  contain  the  term 


i|3c(T)l|g  «  [x^(t)  cos  T  +  Xj(t)  Bln  T]^ 

which  cannot  be  affected  by  u^(t)  and  clearly  does  not  have  a  limit  as  T  -»  oo. 
The  most  impoztant  consequence  of  complete  controllability  la  the  following: 


saanm 


(7.7)  THEOREM.  If  a  syatqn  le  const&nt.  gompletely  eontrolla'ble,  ea\d 

S  =  0,  then  the  limit  (7.I)  al^va  ex  lets. 

By  complete  eontrollAbllity,  V°(x,  t,  T)  may  be  bounded  from  above .  Independent 
of  f.  The  thjorsa  then  follows  immediately  since  V®(x,  t,  T)  is  nondecreaslng  as 
T  -»  00  and  a  bounded,  monotone  sequence  always  converges.  It  is  an  open  quo at ion  at 
present  whether  this  theorem  holds  also  when  8  /  0  (because  then  V°(x,  t,  T)  Is  not 
necessarily  monotone  increasing  T. ) 

Slight  further  arguments  prove  also 

(7*8)  THEOREM.  Let  P(t;  0,  T)  be  a  solution  of  the  riccati  equation  correspond¬ 
ing  to  P(T;  0,  T)  ■  0.  Then  * 


(7.9) 


P»  -  lim  P(t;  0,  T) 
T  -»  00 


always  exists,  and  P»  is  on  equilibrium  state  of  the  riccati  eqmtlon,  l.e., 
dP/dt  »  0  when  P  =  P*. 

(V 

See  (Kalman,  I96IA]. 

We  note  also  that,  whenever  the  limit  (7.I)  exists,  the  following  is  true; 


(7.10)  oiln  (  lim  V(x,  t,  T;  u))  ■=  lim  (min  V(x,  t,  T;  u)  ] 

u  T  -» 00  T  -*  00  u 

lim  V°(x  ,  t,  T). 

T  -♦  00 

The  left-hand  side  is  the  definition  of  optimal  control  for  infinite  terminal  time 
T  o  00.  The  right-hand  side  shows  that  the  min  and  lim  oporatlons  may  be  interchanged; 
in  other  words,  optimal  control  when  T  »  00  can  be  obtained  as  the  limit  of  optimal 
controls  as  T  00. 

The  proof  of  (7*10)  is  almost  immodiate,  appealing  to  the  definition  of  optima¬ 
lity. 

Finally,  let  'jb  observe  that  while  complete  controllability  g’jarantees  the  exist¬ 
ence  of  the  limit  (7.9)  when  S  =  0,  It  may  happen  that  for  other  values  of  S  there 
will  be  a  different  limit. 

Ihe  equilibrium  states  of  the  riccati  equation  are  obtained  by  setting  dP/dt  =  0. 

We  show  that  It  is  possible  to  have  under  complete  controllability  more  than  one 
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e<iulllbrlun  fitate.  Ccnii^r  the  eyiten 

*1  -  »i  +  V 

llT.ii)  *i!  ■  *2  “a 


f,  ■  3f,  +  X_. 
X  ■  1  2 


Here 


H  -  [1  1]. 


'(7*5)#  bbe  eyetem  la  clearly  completely  controllable, 
be  let 

••  ll],  R  -  [n  1 J* 

(«  ' 

Iben  the  zlccatl  equatlom  la 


-dpj^/dt  »  2p^  -  Pn  ^  rL 

> 

(7.12)  -  2p^  -  ^’12^^11  ^22^ 

“^12"  ^22 


It  can  be  proved  [HSLlnan,  I96IC,  ^^iwraple  lU.201  that  on  setting  the  left-hand 
sidea  In'tT.lS)  eqiuil  to  tero,  the  reaultlng  set  of  quadratic  eq'jations  has  precisely 
tvD  nonnegative  d.eflnlte  solutions: 


■3  +'/3 

V3  -  1 


•/5  -  1 " 

3  +>r5 


and 


^1*. 


>\¥fgs 


p(2)  i  i 
P  “  $ 


r  1 4-^/5  1 +/3 


1  +«/3  1  +n/5  j 


is 


nonslngular,  uhlle  P'  '  is  singular. 


8.  UhiguspeBS  of  the  Solution  of  the  Staady-State  Itegu3Ator  Problam. 

To  prove  that  the  stea  dy -^tate  control  law  is  unique,  l.e.,  Indepandent  of  8, 
we  need  a  new  concept,  \iiilch  may  he  regarded  as  the  dual  of  controllablli^y. 

f 

DEFINITION  OF  CQMPIBTS  OBSERYABILITI.  system  (l.lP)  is  said  to  be  coagletels 

obaerrable  if  it  is  possible  to  determine  the  exact  value  of  x(t  )  given  the  values 
of  y(t)  in  a  finite  interraOL  (t  t^)  preceding  t^. 

Bie  analog  of  Uteorem  (7.2)  is  proved  Jjn  [Kalnan,  I96IC,  Lensna  (15.7) )  and  may 

be  stated  as  follows: 


(8.1)  TBEOREM. 


Ls  comt 


obf-'TYable  if  and  only  if  the 


(8-2) 


M(t^,  T)  -  /  »’(t,  T)H’(t)Q(t)H(t)®(t,  T)dt 

% 


is  positive  definite  for  some  T  >  t 


The  analog  of  TSieorem  (7.5)  is! 


(8. 5)  THEOREM.  For  a  constant  system  (l.l-g)  a  necessary  and  sufficient  condi¬ 
tion  for  coBgilete  observability  io 

rank  [H’,  F’H’,  ...,  ]  .  n. 


According  to  this  criterion  the  system  (7. 11)  is  not  con[5>lctely  observable.  Sbus 
the  exaaqple  at  the  end  of  Becticm  7  shews  that  in  the  absence  of  coinplete  obeervs.- 
blllty  we  cannot  expect  In  general  to  have  a  unique  optimal  control  law  In  the  steady- 
state. 
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!I!he  main  result  of  this  cliapter  way  Tse  stated  as  follows ; 


■(S.ll-)  TESC/RSM,  Consider  a  ccostant  system  (*l.l-e).  l.e.,  F,  G,  K,  Q,  R  are 
constant  eatrloes.  Assume  that  the  systoa  Is  coBpletely  controllable  and  completely 
oTaservable .  Bisnt 


(I)  ^tbe  solution  of  the  rlccati  eguatlon  starting  at  any  nonneagtlye  definite 
matrix  8  converges  exponentially  to  a  unique,  positive  definite  matrix  P  m 

t  -♦ -oo  (or  T-»oo). 

(II)  Pie  optimal  control  law  for  T  =  6o  la  constant  and  the  optimal  regulator  Is 

asymptotically  stable. 

Uhls  theorem  can  be  generalized  in  a  natural  and  straightforward  wiay  also  to 
nonconstant  (time •varying)  systems.  The  precise  ebatement  of  the  results  is  more  com¬ 
plicated.  >'or  these  statements  and  the  proofs  the  reader  Is  referred  to  [Kalman, 

196IA  and  1561c]. 

,  The  fact  that  under  conditions  of  complete  controllability  and  conplete  observ¬ 
ability  the  optimal  system  Is  stable  Is  not  a  trivlsLllty  since  the  formulation  of 
the  optimization  problem  In  Section  2  did  not  include  this  requirement.  Nor  does 
stability  of  the  <9tlfflal  system  follow  in  general.  For  instance,  if  ve  taXo  the 
matrix  of  Section  J,  we  find  the  corresponding  infinitesimal  transition 

matrix  of  the  steady-state  optimal  system  is 


r 


o 


F  -  gR 


-1  -  V3  " 

1  -  ^3 


whose  eigenvalues  are 


> 

“1 


-  ?  . 


> 

2 


-2. 


Thus  the  optimal  system  for  T  ■=>  00  is  unstable  if  we  choose 

S  = 
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But  the  deeper  algnlficance  of  Theorem  (S.  Ij-)  lies  in  the  result  that  every 
solutKai  of  the  riccati  eouatioti  starting  at  a  nonnegatlve  definite  Initial  value 
converges  to  Pf  moreover,  convergence  is  exponential.  This  means  that  the  riccati 
equation  provides  a  feasible  computation  procedure  for  obtaining  the  optinal  eyatete 
vhlch  is  not  likely  to  be  affected  by  roxmdoff  errarn.  Note  that,  according  to  the 
theorem,  one  could  have  obtained  jP  by  settlitg  the  left-hand  aide  of  (1.6)  equal 
to  zero  and  solving  the  reau.ltlng  set  of  simultaneous  quadratic  a3.gebralG  equations 
in  the  elements  of  P.  Rils  procedure  can  Indeed  be  carried  out  in  elmpL?  cases 
[Kalman,  I96IC,  Sect,  lU]  but  when  the  order  of  the  system  becomes  larger  than  2, 
the  approach  via  the  riccati  eqmtion  is  likely  to  be  appreciably  more  efficicnl. 

•  9*  Some  Important  Inequal It la 8. 

From  the  point  of  view  of  practical  numerical  computation  It  in  of  course  by 
no  meems  enoiigh  to  know  that  the  solution  of  the  riccati  equation  converges  expon- 
entially,  one  must  have  also  an  estimate  of  the  speed  of  convergence. 

This  aspect  of  the  theory  is  not  yet  In  a  definitive  form.  We  shall  confine 
ourselves  therefore  to  the  statement  of  the  major  results  'co  date.  Proofs  may  be 
found  in  [Kalaan,  I96IA  and  I96IC]. 

If  A,  B  are  syuunetrlc  matrices,  let  us  use  the  notation  A  >  B  [A  a  Bl  to 
signify  the  fact  that  A  -  B  is  positive  [nonnegative]  definite. 

We  assume  (1.1-2)  is  constant,  completely  controLlablo  and  completely  obser¬ 
vable  . 

15ie  desired  inequalities  are  then  as  follows: 

(9.1)  0  <  P(t)  S  W"^(t^,  t)  +  M(t^,  t), 

(9.2)  P"^(t)  S  M"^(t^,  t)  +  W(t^,  t),  t  >  t^ 

,  X  ,  (M^(t  ,  t,,jj 

(9.5)  P(t)  -  P(t  )  a  - - I 

°  U  tr  M^(t^,  t)tr  W(t^,  t) 

where  denotes  the  smallest  eigeisvalee  of  a  symmetric  matrix  A. 

These  Inequalities  are  useful  in  guiding  the  choice  of  numerical  values  of  Q 
and  R. 
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(B)  CONi^ENTIONAL  BLOCK  DIAGRAM  OF  LINEAR  DYNAMICAL  SYSTEM 


Fia  1  LINEAR  DYNAMICAL  SYSTEM 
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lu;) 


FIG,  2  OPTIMAL  NOISE-FREE  REGUUTOR 
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STATE-SPACE  OF  A  RICCATI  EQUATION 


Chapter  5. 


A  THlRD-ORISfu?  OPTIMAL  REOHIATOR  PROBLEM 


1.  Introduction. 

f 

We  Shall  discuss  'u  this  chapter  the  noise -free  optiiBa.1  regulator  probiea  In 
a  socclal  case  vhen  the  control  ohleet.  ic  cf  the  third  order  (n  =  3)-  On  the  ono 
hand,  this  example  is  simple  enough  to  be  treatable  in  part  by  analytic  methods;  on 
the  other  hand,  the  example  is  complicated  enough  to  illustrate  •warious  problems  en¬ 
countered  In  numerical  computation. 

2.  Definition  of  Control  Ob.lect;  Transition  Matrix. 

Ihe  defining  matrices  in  equation  (l.l)  of  Chapter  2  are  taken  as 


0 

100 

o"' 

r~  — s 

1 

(2.1) 

F  - 

1 

-100 

0 

1 

and 

G  ■« 

0 

0 

-1 

0  1 

0 

J 

_ 

The  matrla  H 

will  remain  undefined 

for  the 

moment. 

The  (conventional)  block  diagram  of  the  control  object  is  sricnm  in  Figure  1. 
Sot‘j  that  the  element  values  of  F  e.nd  0  may  be  read  off  by  Inspect  Ion  from  the 

figure. 

We  now  compute  the  transition  matrix  corresponding  to  F  given  by  (2.1).  For 
simplicity  we  write  *(t)  -  'I'(t.  0).  By  (1-7),  of  Chapter  2  <?(t)  satinfies  the 

reijations 

(2-2)  a«(t)/dt  -  FO-Ct),  i'Co)  -  I. 

^Baking  laylace  transforms  on  both  sides,  vo  have,  just  as  In  the  scalar  case. 
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(2.3) 


(»I  >-  P)#(s)  -  I 


One  can  ecxiqpute  •{a)  by  aolviiig  (2,3).  Tbis  is  quite  po^pllcated^  Ucwever. 
« 

A  aljqpler  aethod  is  thlo: 

We  obaerve  that  by  (2.2)  i"  th«  tranafer  fiihcblon  from  tn«  Input  to 

the  J-th  integrator  to  the  output  of  tho  1-th  Integietor  in  Figure  1.  Utilizing 
Muon's  loop  rule  [)#Mion,  1556],  we  can  easily  calculate  these  transfer  functions 
SLi«5.  obtain 


•(a)  - 


8(8*^+  10,001) 


8  1 


-100b 

100 


lOOe 

r> 

e" 

-s 


IOC 

8 

B^+  10,000 


which  cheek"  with  (2.5)?  Teklng  the  inverse  laplace  transform  of  each  element  of 

A(r)i  we  finally  get 


♦(t) 


2 


(0 


p 

1  +  (a)‘-l)coB  ojt 
-100  CD  sin  oat 
100(l-CO6  cut) 


100  u)  sin  oot 

2  . 

(D  COB  OJt 

-CD  sin  cut 


100(l-CO8  cut) 


CD  sin  cDt 


10  +  COG  a)t 


where  ca  “  10.001, 


3.  Controllability. 

We  can  use  Theor?*m  (j.*'!  of  Chapt^-**  2  to  check  whether  the  system  (2.1)  is  coa-.- 
pletely  controllable.  The  answer  is  in  the  acffirmatlve,  for  the  matrix 


1 

0 

0 


0  10,000' 

-100  0 

0  -100 


k6 


[0,  TO,  F^O] 


has  jisuok  3. 

this  result  Is  merely  <iuall'tative.  To  get  u  qtueuitltative  emevcr  aa  to  hew 
effectively  ccmtrol  car  be  epplled>  ve  niiat  conrowte  the  cortro  Xl&lDl.a,x vj*"  .iu«'t3rl.x 
given  by  (7*3)  3f  Chapter  2,  and  find  Its  inverse  (which  always  exists  by  ccsspleto 
controllability)  to  see  how  much  energy  Is  needed  to  take  the  various  states  to  zero. 

We  could  cooipute  W  by  direct  integration,  since  the  integrands  in  (7.3)  of 
C?haptar  P.  would  involve  only  simple  trigonooetrlc  functions.  But  this  teak  ie 
exceedingly  tedioiia.  Calculating  crxdely,  we  see  thet  if  T  ~  t  is  several  times 
larger  than  the  period  {tHr/<D  ■  6  X  10"')  of  5)(t),  then  the  amount  of  en.5rgy 
req,ulred  to  take  the  states 


5 

to  aero  is  about  5  X  10  times  emeller  than  the  energy  req^uired  to  take 


to  zero. 

In  pj'actlce,  the  matrix  W  is  obtained  by  computing  the  solution  of  the  differ¬ 
ential  equation  (7*^)  of  Chapter  2  by  meaca  of  the  nathods  discussed  in  Chapter  IJ'. - 
Taking  R  ■  [1],  we  get  the  matrix 

4,9765  O.UI26 

w(o,  1)  »  0.0126  5.0201 

-0.0503  0.0000 

whose  Inverse  Is 

'0.0003 

W"^(0,  1)  »  0.0000 

,  0.0101 

Ut 


0.0000  O.OIOj  ‘ 

0.0002  0.0000  X  io\ 

0.0000  1.0001 


-0.0503 

0.0000  X  10"^ 

0.0015 


The  numerical  petmlts  thus  confirm  the  eaa-lier  qualitative  concluaione. 


h.  First  Attempt  at  Design. 

We  assume  that  the  prinafy  objective  of  control  la  to  assure  that  x.  is 
smUJLiAt  all  times.  Therefore  vc  set 


(^.1}  H  -  [1  0  Oj. 

It  is  a  good  idea  to  check  immediately  whether  with  this  choice  of  h  the  system  is 
completely  observable-  In  view  of  Theorem  (8.3)  of  Chapter  2,  complete  obscrva  - 
bility  is  equivalent  to 


det[HS  F'HS  F'^‘] 


0  10,000  ‘ 

-100  0 
0  10C> 


/  0. 


Hence  H  given  by  (U.l)  assures  complete  observability. 

Guided  by  the  aujalysls  of  Section  5,  Chapter  2,  we  now  wish  to  choose  the  ratio 
large  In  order  to  assure  an  adequate  degree  of  stability  (in  the  present  cane 
both  Q  and  R  are  1x1  matrices,  l.e.,  scalars).  Suppose  we  let  ^/r  ™  10^, 
Moreover^  in  view  of  the  method  for  congputlng  the.  rlccati  equation  explained  In 
Chapter  3  1,  it  is  best  to  take  Q  ■  r"^  ■  10^. 

In  view  of  the  discussion  of  Section  k  of  Chapter  2,  the  steady-state  value 
of  P  can  be  obtained  by  setting  dP/dt  -  0  in  the  ilccati  equation  (l+.o).  More¬ 
over,  this  solution  is  always  unique.  We  observe  that  if  P  I,  then  dP/dt  =  0. 
Hence  P  -  I  Is  the  steady-state  solution  of  the  riccati  equation.  Therefore, 


K  -  IT^O’P  -  lO^G’-  [100  0  0] 

is  the  ‘OptiJBal.  gain.  The  Infinitesimal  transition  matrix  of  the  closed-loop  optimal 
system  ist 
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p®-r-aR»F  -  10  ao‘  * 


+  100 


“  1 
+  1  I 


The  eigenvalues  of  this  natrlx  are  given  by 


(4.3) 


"  -  .01000 

/  X^  ,  «  -  49.9950  t  1  86.6054, 

r  2,3 


which  ijhows  that  the  opt/jual  closed  loop  system  le  very  poorly  damped. 

The  explanation  for  the  poor  damping  is  the  follofcdng*  The  criterion  of  optimal¬ 
ity  requires  only  that  be  quiclcly  reduced  to  a  sinall  value.  This  d.oe«  indeed 

happen,  since  the  first  row  of  the  optimal  closed-loop  transition  matrix  la  given  by 


9ll(t)  -  10"V*°^*  +  1.154e"^^*^^^  sin  (86.605t  +  2.095), 

cPlgCt)  -  -lO'^e''^^*^  +  1.15Ue‘^^*^^^  sin  (86.605t  +  10"^), 
<p^^(t)  -  lO’^e”*®^^  -  .0115e‘^^*^‘^^  sin  (86.605t  +  1.047). 


This  shows  the  effect  of  unit  initial  conditions  in  x^,  on 

On.  the  other  hand,  the  criterion  of  optimality  does  not  require  good  control 

over  X-.  Since  (see  the  discussion  of  controllability)  x,  ic  very  weakly  coupled 

3 

to  X,  ,  the  good  transient  response  of  does  not  bring  about  a  aimllarly  good 

treuoBlent  response  in  x_.  Another  way  of  saying  the  same  thing  Ip  tliut  the  control 

3 

eneigy  is  used  more  effectively  in  reducing  x^  than  in  reducing  x  ,  because  x 

X  2  j  j 

does  not  enter  directly  In  the  error  criterion  |(.Ix||q  and  the  amount  of  energy  re-- 
quii-ed  to  quickly  take  x^  to  zero  is  enormous. 

It  should  be  noted  that, in  accordance  with  the  general  theorem  ( 8.  4)  in  Chapter 
2,  the  closed-loop  optimal  system  is  asyraptotlcally  stable. 
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5. 

ilotiTfeted  ty  first  set  of  results,  ve  nw  let 


(5.1) 


E  - 


10  0 
0  •  0  1 


sbA 


(5.2) 


Q  - 


100  0  1 
0  100 


J’’ " 


In  other  wox^l8,  we  velghterrors  In  and  equally'’. 

The  oheejrvebllity  lAtrlx  corresponding  to  (5.I)  and  (5.2)  was  obtained  by  compit- 


ing  the  solution  of  the  rlccmti  equation  (U.  6)  of  Chapter  2  wi  th  R  ^  r  0.  We  found. 


M(0,  1)  - 


4.9775 

0.0126 

0.0502 


0.0126 

5.0204 

-0.0001 


0.0502  ] 
-0.0001 
9.9^95 


X  10  , 


and 


(5.3) 


M"^(0,  1)  - 


r  2.0090 

-0.0050 

-0.0101 


-0,0050 

1.9939 

o.Oooo 


-0.0101 

0.0000 

1.0001 


X  10' 


The  steady-state  raluc  of  P  was  obtained  from  machine  calculatioms  au 


(5.4) 


0.0101 

-0.0001 

0.0099 


-0.0001 

0.0101 

-0.0100 


0.0099 

-0.0100 

1,0093 


X  10^. 


This  gave  an  optimal  steady-state  gain  matrix 


50 


(5.5) 


K  -  [1.0099 


-0.0099  0.90993  x  10' 


and  an  Infinites Inial  tranaltion  aatrlx  of  the  clo«ed-loop  optljaal  oyntein 


(5.6) 


-1.0099  1.0099 


F  -  r  -  (K  - 


I 

-1. 


-O.Oi 


.0.9&99 


The  rilgensmlueo  (5.6)  are 


(5.7) 


X,  -  -  .9999, 


^,3  “  ■'  ^  86.605X, 


which  shotra  that  we  hare  iiKprored  the  daagpiag  by  a  factor  of  about  ,100  by  giving 
equal  weight  with  in  the  error  criterion.  Defined  by  H  and  tiie  .msw 

error  criterion  hat  forced  the  eyste*  to  distribute  the  contro.l  energy  baitter  be¬ 
tween  x.^  and  x^.  This  iaprovosertt  has  not  been  B»de,  howerer,  without  a  consider- 
able  increase  in  required  control  energy,  even  though  the  first  ccaaporasut  of  the 
K  matrix  remains  ossertially  the  sasM.  Ibtc  also  that  is  100  timoe  larger  in 

this  case  than  in  the  previous  section^ 

Finally,  observe  that  control  over  aid  Xg  Is  virtually  unchanged,  and 

tbs  costplex  eigenvalues  of  P^  hsrre  remained  the  same. 


Another  design  was  inrestlgatsd,  setting 


(6.1) 


f” 

-  I  0 

[o 


im?  r»  fi 


0  500 


Krror  in  x^  are  weighted  five  tines  more  than  errors  in  We  ag^ain  have  com¬ 

plete  obeervnblllty. 
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!lJhe  obaerrabllity  aatrls  was  found  to  be 


(6*2) 


::^o,  1) 


r  0.J»98!» 
0.0015 
L  0.0452 


0.0015 

0.5025 

-0.0001 


0.0452 

-0.0001 

4.9991 


X  10^  .• 


notice  that  the  tenas  In  the  third  row  aad  third  column  (l.e.,  terms  associated  with 
z^)  are  auch  larger  than  In  (5<2}. 

The  ateady-state  ralue  of  P  was  obtained  froai  machine  chIcxiI  ;tlrns  and  It  was 
obaerred  that  P  courerged  more  qulc!cly  than  It  did  vmder  the  condltlom.  of  Section 

5: 


(6.5) 


0.0102  -0.0000  0.0225 
-0,0002  0.0102  -0.0228 
0.0225  -0,0228  2.2862 


X  10 


Ve  see  that  all  tenaa  associated  with 
than  In  (5*4). 


are  (approrlaately)  </5  tljases  larger 


optimal  steady-state  gain  matrix  was  found  to  be 


(6.M 


E  -  [1.0223  -0.0225  2.2259 J  X  10'  . 


The  infinitesimal  transition  matrix  of  the  cloued-loop  optimal  system  was  found 
to  be: 


(6.5) 


-  F  -  GK 


-1,0225  1.0225 

-1.0000  0. 

0.  -0.0100 


-2.2259 

0.0100 

0. 


2 

X  30  . 


The  eigenvalues  of  (6.5)  mtrlx  are 


(6.6) 


-  -  2.2555, 

Xg^^  -  -  49.9949  +  1  86.6399. 


TLa  followlxig  are  soMe  noteworthy  of  this  example* 
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(a)  TSae  eleoente  and  in  (6.4)  are  larger  by  a  factor  of 

then  correapondlng  elements  of  (5*5)*  Tbis  Is  due  to  the  change  in 

(B)  Despite  this  change,  le  aov  still  only  .02  times  vhich  btta 

remained  unchanged.  In  other  words,  the^^^  is  essentially  no  chjsiige  te  the  control 
over  and  Xg.  Hence  the  complex  pair  of  eigenvalues  in  (6.6)  remains  about 
the  same  as  (5.7). 

(C)  The  ubiquitous  factor  of  -/j  is  to  be  expected  from  the  scalar  analysis 
in  Chapter  2;  see  cquabicnu  (5*6)  and  (5-7).' 


7.  Fourth  Design. 
Finally,  we  took 


(Y.i) 


H'CJH  - 


100  0  0 

0  200  0 
0  0  500 


Again  we  have  complete  ohservahlllty  with 


(7.2)  M(0,1) 


1.5025 

-0.0013 

0.0352 


-0.0015 

1. 4977 

-0.0000 


0.0552  1 


-0.0000 


4.9992  J 


X  10 


Tbe  riccati  equation  converged  to  a  ateady-etate  value  of  F  more  slowl^^  tnan 
.n  the  third  design  (Sect.  6),  indicating  that  the  laryeut  eigenvalue  of  the  closed 
loop  system  is  somewhat  closer  to  zero. 

The  steady-dtate  gain  matrix 

(7.3)  K=  [1.5826  -0.7523  2.2203]  X  10^. 
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tPhe  Iftflrdteflinail  translticn  matrix  of  the  eloBed  loop  optimal  flyatem  was 


(7.U )  F®  ■  p  -  erif  - 


i.5826 
-  1.00 
0 


1.7523 

0 

-0.0100 


-2.2205 

0.0100 

0 


X  10®. 


5b*»  elgenyalues  of  this  matrix  (7.U)  aj>^' 


(7.5) 


-  -i.2910 

f 

-c  -  -78.1>830  +  i  211.2952 

2,3 


Dm  Blse  of  the  real  eigenvalue  accords  with  the  qualitative  prediction  made 
from  the  rate  of  convergence  of  the  riccatl  equation. 

(Juivlitatlvely  the  Bhlfts  In  the  eigenvalue e  could  be  predicted  on  the  basiy  of 
Btorfi  control  energy  being  put  into  the  -  Xg  loop  at  the  expense  of  x^  control. 

Quantitatively  the  plctuire  Ib  considerably  more  complicated  than  It  vau  in  the  one 
dimensional  system  analyzed  In  Chapter  2.  For  instance,  it  would  appear  heuristi- 
cally  inviting  to  assume  that  x^  and  Xg  are  so  tightly  coupled  that 


H'QH  = 


f 100  0  0 

0  200  0 

0  0  500 


18  the  same  au 


0  0 
0  0 
0  500 


and  such  an  asoumption  is  possible  to  maintain  about  the  obseivabllity  matrix  wbere 
a  factor  of  three  appears  in  the  x^^  -  Xg  terms.  But  tills  point  of  view  lo  too 
naive  to  account  for  the  changes  in  P  and  R. 


A  careful  analysle  of  the  mear,«  of  applyins  inequalitieB  of  Chapter  2  ib 
required  in  order  to  obtain  Information  about  Q  and  R  in  termn  of  parametere 
more  familiar  to  the  engineer,  such  ao  time  constants  and  frequencies. 
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ChAptor 


oPTiMAi.  nimsKiac  !rasosar 

MotlTmtion  of  AaBUHPtlons. 

Another  problem  of  optimization  of  lttpertanc:j  for  the  stuSy  of  adaptlm  eyatemfl 
is  that  of  Btatistical  eetlmtlon  theory  or  esnemllzed  Wiener  filtering.  In  this 
problem  it  is  usually  asBumed  that  one  oboervoo  a  signal  In  the  presence  of  acldltlve 
noise  and  one  desires  to  find  a  "best  eetlnate''  of  the  signal  by  linear  operations 
on  the  observationa. 

In  relation  to  control  theory,  the  asauniptlons  are  stated  In  a  aoiseubat  differ¬ 
ent  (but  by  and  large  equivalent)  form.  We  assume  that  the  state  x  of  the  control 
object  cannot  be  obaerved  directly.  We  can,  however,  measure  come  linear  combinations 
of  the  state  variables.  These  measurements  are  denoted  by  the  vector  y.  !nie  measure¬ 
ments  are  not  made  with  perfect  accuracy,  so  that  actually  we  observe  a  vector  z 
which  is  the  sum  of  y  and  a  voctor  v  representing  measui'eiaent  errors  or  noise. 

In  Bidditlon,  It  Is  also  assumed  that  the  control  object  Is  subject  to  certain  random 
disturbances  w  acting  on  It, 

In  accordance  with  the  discussion  in  3ect.  2,  Chapter  1,  these  aocumptlons,  com¬ 
bined  with  linearity,  yield  the  equations 

(l.l)  dx/dt  ■  F(t)  X  +  G(t)w(t) 

(l.?)  z(t)  =>  y(t)  +  -"-(t)  -  K(t)x(t)  +  v(t) 

where  w(t)  and  v(t)  ore  random  processes  (see  below).  In  tbs  first  equatloii. 
omit  a  term  Involving  u(t)  since  we  are  not  concerned  here  with  the  control  problem. 
This  term  will  reappear  again  in  Chapter  5- 

Alternately,  one  c.nn  Interpret  these  equations  in  the  folxowing  V’rras;  We  are 
given  a  random  process  x(t)  and  a  related  random  process  z(t).  Values  of  z(t} 
are  ohservod  over  a  certain  Inteiral  of  time.  On  the  basis  of  these  obaervatlcms, 
we  wish  to  estimate  the  value  of  x(tj^),  where  t^  is  some  arbitrarily  -hoson 
Instant  of  time.  It  can  be  shown  that  if  x(t)  is  a  marlr.oTlan  and  gausolan  raiidora 
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prooei»y  It  0*01  aliiiqn  bd  r«pr«s«nt«4.  by  e  soheaw  such  aa  (1.1-2).  See  [Kalnan, 
Sect.  7l. 

Vis  oalce  the  further  aeauaptloa  that  7(t)  agd  v(t)  are‘  tcauiBian  uhlte-noige 
proeeeeee  Independent  of  each  other,  ^ey  axe  apecifled  mathematically  by  their 
curariancs  aatriceai 


(i.3) 


'•*T(t)T'(T)  -  (!SV^(t)Vj(T)]  •  R(t)B(t  -  t), 

\  -  5(t)6(t  -t). 

% 

S  »jr(t)T’(T)  -  0  for  all  t,  t, 

(  '  ■■ 

L  Sv(t)  ■  Bv(t)  «  0  for  all  t, 


ithere  £  '.'epreeenta  the  tnathematlcal  expectation,  &(t  -  t)  ie  the  Dirac  delta  func¬ 
tion,  5,  ft  are  posltire  definite  matrixes.  Bie  case  of  nonwhite  nolbe  can  be  re¬ 
duced  to  this  foraulctloo  by  a  change  of  variablee. 

■We  shall  often  refer  to  (1.1-2)  as  the  model  of  the  signal  process.  Kie  matrix 
block  dis^praa  for  the  model  Is  seen  in  Fig.  1. 

A  {Bueh  more  detailed  discussion  of  the  subject  of  this  chapter  may  be  found  in 
[Xaiman,  19610]. 

2.  Statement  of  the  Filtering  Problem. 

Bie  filtering  problem  can  be  stated  as  follows:  Determine  a  linear  operator  on 
the  set  of  obmerTatlons  {z(T)|Te  [t^^  t])  whose  valua  x(tj^|t)  at  tlcae  t^  has 


(I)  Ex(t^1t)  -  Ex(t^)  , 

(II)  Ell?(t  lt)|l2  ,  Z  b  FSc  S 

"  1,J-1 


mlnlQn:a  {B  o^v  powJ.ti^ce  definite 

matrix! . 


Above  we  used  the  abbreviation 


(2.1) 


x(t^lt)  »  x(t^)  -  $(t^|t) 


for  the  error  In  the  estimate  x(tj^|t).  The  observations  of  z  otax’t  at.  time 
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(liilich  is  taken  as  fixed),  and  end  at  time  t  (vhich  is  taken  as  a  running  para- 
aeter) . 

Thus  x(t^!t)  is  to  be  taibiaaed.  aininiuia  •yarianee  astiaator  of  x(t);  that  is, 
3^(tj^lt)  minimizes  tlie  average  value  of  the  squares  of  ttie  error.  One  of  the  interes 
tlnit  properties  of  the  estimator  1^(tj^|t)  is  that  the  best  estimator  of  the  scalar 
quantity 

n 

(2.2)  ^  *1^1'*' 

lail 

t\ums  out  to  be 

a’x{tj^|t). 

f 

3.  Solution  of  Filtering  Problem. 

■  By  a  rather  involved  eurgument  glTr<n  in  detail  In  [Kalman -Bucy  19^1;  KaLaari  196IC 
it  can  be  shown  that  j[(t|t)  is  the  output  of  a  dynatalcal  system  similar  to  (l.l) 
whose  input  consists  of  the  obsez^atlons  z(t): 

(?.l)  -  P(t)x(t|t)  +  K(t)[z(t)  -  H(t)5[(t|t)]. 

The  dynamical  system  (j.  1)  can  be  physically  realized  by  a  feedback  system  as  shown 
in  Pig.  2. 

It  can  be  shown  that  K(t),  the  gain  of  t^e  optimal  filter,  Ic  dctcraiiE  d.  by 
the  covariance  matrix  of  the  errors  of  the  cq>tlfflal  filter.  In  fact,  if 

I^c(t  |t)?(t|t)  »  E(t) 

then 

(5.2)  K(t)  -  E(t)H’(t)i'^(t) 

Further,  it  can  be  shown  [Kalasn-lhjcy,  I96IJ  RHiJian  19610]  that  r.(t)  is  deter 
mlaod  as  the  solution  of  the  follbwing  tiatrlx  riocati  equation: 

(5.5)  ^  «  P(t)E  +  EF»(t)  -  EH'(t)§:^-"(t)H(t)E  +  5.(t) 
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vhere 


E(t^)  -  E>c(tjx’(t^). 

To  avoid  confuBion  ‘betwaen  (3>5)  ®*id  th*  matrix  riccsti  equations  of  Chapter  2, 
WQ  shall  uaualiy  refer  to  {5a  3)  aa  tha  vaL-janee  epgation. 

Notice  that  this  equation,  is  solved  forward  in  time.  By  solving  (5.5)  and  then 
using  (5»2)  the  optimal  filter  (3.I)  coa5)letely  specified 

U.  Duality  Relations.  s 

It  should  be  noted  that  the  solutions  to  the  regulator  problem  and  the  f liter ir^ 

problem:  are  quite  similar:  in  each  case  the  problem  reduces  to  the  solution  of  a 

matrix  rlccati  equation.  Actually  nmch  more  Is  true.  To  every  filtering  problem 

there  corresponds  a  "dual"  control  problem  so  that  the  sasie  rlccati  equation  prcvlder. 

the- answer  to  both  problems.  Ihe  "duality  relations"  may  !>“  stated  explicitly  as 

follows: 

«  ‘ 


Filterina 

Control 

L(t) 

—> 

P(t) 

F 

<— 

— > 

F' 

G 

<— 

“> 

U» 

U 

<— 

— > 

G' 

t 

0 

<— 

— > 

T 

5 

<— 

— > 

Q 

ii 

<— 

-> 

R 

This  shows  in  piurtlcular  that  the  conditions  of  complete  controllability  and  com¬ 
plete  observability  are  duals  of  one  another.  A  cocapletrly  control  table  dynt-Mc  sys¬ 
tem  of  a  control  problem  la  the  dual  of  &  comp]  obserrable  dynamical  syst&m  of 

a  filtering  problem.  Bonce  the  existence  and  uniqueness  theorems  in  Chapter  2  are 
valid  also  for  the  fllterlrg  problem,  with  the  conditions  dualized  and  the  conclv- 
slons  now  pertaining  to  th»»’  optimal  filter  rather  tlian  the  optimal  cmtrol  system. 

Each  counter-example  of  Chapter  2  would  serve,  after  It  Is  dualized,  as  a  couriter-exnffipie 
for  a  filtering  theorem. 
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•5.  Bxample  of  a  Filtering  Pro^em. 

Bii*  follwin*  siwi-Jla.l  c»(i«  of  f  11  taring  proWam  vlll  b«  oonaldared  in  datall 
to  llluatrato  the  appllaatlon  of  tha  general  theory. 

The  Blgnad  proceas  Is  given  hy  the  differential  eqoatlon 


(5.1) 


dx^ 

dt  "  ''i' 


the  observed  signal  is  . 

(5,2)  x^  +  vy 


In  accordance  with  (l.)),  It  will  he  aosumcd  that 


(5.3) 


EWi(t)Wi(T)  =  q3jL&(t  -  t) 

EVi(t)v^(T)  =  r^&(t  +  t) 
KxJ'd^)  -  -  a 


or 


F  -  If^^],  G  -  [Ij,  H  -  [n,  5  -  R  »  [r^^]. 

Specializing  (5.1)  to  this  case,  the  eqxiatton  of  motlun  of  the  optimal  filter  is 


(5-M 


dx-^(tlt) 

dt 


o^d) 


U 


[z^(t)  -  x^(tit)  1 


The  block  diagram  of  the  optioual  fii.ter  ir.  ahnvm  lu  Fig.  5. 

®ie  solution  of  (5.3)  in  this  case  can  he  found  by  separatloti  of  variables  and 
integrating.  Ihe  end  result  Is 
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The  solution  of  the  stoady-state  filtering  problem  (t^  ■  oo),  the  conven¬ 
tional  Wiener  problem,  exists  since  the  model  is  coapletciy  controllable  and  com¬ 
pletely  observable.  The  solution  of  the  Wiener  problem  is  given  bj' 


f 


(5.6)  llm  OTi(t) 
t  -♦  +00 


(f  +  ^/  +  f^  )r 

'^11  ^  r^  ^11^^11. 


11 


/ 


'11 


=r^  +  f  ^  -  f 

"'ll  ’'u. 


This  is  a  veil -known  result  of  the  conventional  theory. 

Notice  that  the  optimal  filter  is  stable  regardless  of  whether  the  signal  pro¬ 
cess  was  stable  or  not. 

If  f^  <  <  then  the  time  constant  of  the  optinmn  filter  is  at  most 

1 

I  ,  %ihlch  shows  that  the  less  noise  power  In  relation  to  signal  power  the  faster 

the  filtering  loop.  Hence  the  time  constaint  of  the  optimal  filter  depends  directly 
on  the  signal  to  noise  ratio-  Since  the  filtering  problem  is  the  dual  of  the  control 
problem,  all  of  the  extensive  dleciiasloii  In  Chapter  2  is  relevant  also  to  the  fllter- 
Ing  problem. 
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Ghfttit-er  6 


THE  ADAPTIVE  CORTHOL  PHDBUEM 


1.  Orientation. 

It  Is  nov  quite  clear  hov  Fig.  2  of  Chapter  1  is  to  be  jtaieiiy  Ve  aesvunp 

that  the  equations  of  motion  are  giyen  in  form  introduced  in  Chapter  1.  We 
the  filtering  problem  first,  yielding  the  boit  marked  "state  estimator".  Then,  as 
discussed  in  the  preceding  chapter,  we  form  the  "optimal  controller"  by  cperatjjig 
on  1k,(t).  lEaiB  will  be  a  linear  operation,  represented  by  the  matrix  K^Ct). 

Aseume  now  that  we  hare  a  means  of  measuring  the  values  of  the  iwitrlcec, 

F,  0,  H,  q,  R,  The  estimates  of  thece  parometcrE  will  form  the  icomlng  statea. 

She  other  pamitct'^ee  of  the  problem,  nameiy  3,  Q,  R  specifying  the  quadratic  i>cr- 
formance  index  axe  usually  given  exactly. 

*  With  these  estiBUitea,  wo  coopote  the  solutions  of  the  two  rolovnnt  riccati 
equations  of  the  filter  and  roguLator  problem,  closing  the  "adaptive"  loop. 

Of  course,  nc  claim  can  be  made  at  this  time  that  such  a  procedure  is  optimal. 

It  is  pmbably  not.  But  the  coohined  problem  of  instantanttiTusly  beat  control  and  best 
estlaation  of  the  structural  parameters  Is  too  difficult  nt  prcHent  to  be  seriously 
studied. 


2.  ideal  Adaptatj.on, 

the  val'w-es  of  F,  Q,  H,  J,  5,  ^  by  observing  the  Bystca  output  ^ft),  Ihen  cm«:  cev’M 
deslgn  a  controller  on  the  basis  of  the  theory  of  Chapt#.»r  5*  cootbinatiaa  of  tho 

general  ccntrollcr  and  this  ideal  learulng  will  bs!;  callpd.  sui  .idg~x~»l  adapt Itc 

syatem.  Obviounl^-,  it  will  have  tho  best  porfomanoe  u.. 

of  the  theory  of  the  general  control  problms  prexwsntod  In  Chapter  5,  tho  perforrciacu- 
of  the  ideal  adaptive  ayatem  In  a  given  envlromcsnt  can  be  determined  exactly  —  rhla 
ia  Just  the  general  time  varying  control  problem. 

The  consiept  of  an  Ideal  adaptive  eystea  has  two  major  practical  uses: 
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The  performance  Indax  la  nov  defined  to  he  the  average  -value  of  V,  denpted  hy  EV. 
The  average  le  -taken  irlth  respect  to  tbo  probability  dlatrlbutlon  of  w. 

The  general  control  problem  Is  -then  the  followlngi  Find  a  control  u(t)  such 
that  EV  Ig  mlnimlyied. 

In  Chapter  2,  the  optimal  u(t)  depended  only  on  the  initial  state.  Here  this 
is  no  longer  true,  because  -the  effect  of  w  cannot  bo  piedicted  at  the  beginning  of 
the  control  pix^esa.  As  euSdltlonai  values  of  tbe  state  are  m&iiQured,  0101-0  irforoiation 
is  ob-talned  and  this  Information  must  somehow  be  utilized  In  computing  the  optimal 
u(t). 

f 

To  define  the  problem  precisely,  we  must  -therefore  slIbo  add  the  following: 

Control  must  be  based  on  the  actually  observed  values  of  r.(t)  In  the  intgrya]. 

(V  t). 

3.  Solution  of  the  General  Control  Problem. 

In  this  section  we  shall  gl-ve  the  form  of  the  general  flolutloii;  details  will  be 
omi^tted,  since  -the  -theory  Is  not  yet  comple-te. 

Die  best  estimator  ^(t|t)  of  x(t)  Is  orthogonal  to  the  error  of  estiuiation 

x(t|t)j  hence 

It  can  be  shown  that  x(t|t)  and  x(t|t)  satisfy  th«^  dlffei'entlal.  oquatlone 

(5»2)  ”  5'(*)x{tit)  +  K^(t)[H(t)x(t  jt)  +  v(t)  ]  i-  Q(t)u(ty 

and 

(3.5)  -  F(t)i(tlt)  .  K^(t)[a(t)x(tit)  +  T(t)]  -H  w(t) 


Op  -the  basis  of  equation  ^.1  )lt  foUowa  that 


vhere  ^  Is  the  motion  of  (5.2)  and  ^  it»  action  of  (5-3).  Froni  the  fora  of  (5.3-2i.) 
we  see  that  the  problem  splits  Into  two  piurts^ 

(a)  Sotlaate  x(t)  by  filtering  theory. 

(B)  Corjtrol  the  system  defined  by  (5.2)  according  to  ti.f?  noi8e"fj?0c  regulator 
theory.  f 

This  "decoupling"  of  the  problem  into  the  two  portu  previcuBiy  di»cw?oed  jp  dx«; 
to  the  linearity  and  the  fact  that  the  random  forcing  term  tn  (5.2)  is  a.  vrhlte-noiBe 
process  with  xcro  mean.  Since  such  a  process  l3  complel^ly  unpreAictable,  It  cannrt 
be  taken  into  account  In  computing  the  optln»l  control  lav.  Jsi  other  woi^s.  the  solu¬ 
tion  of  the  regulator  problem  when  the  atate  can  be  exact^ly  and  ijustantancoucly  tuec-Bured 
le  the  sane  with  or  wlth'-mt  a  vhlte-noise  type  of  fcrclng  tern, 

*  The  canonical  form  of  the  optimal  control  ayuten  In  the  general,  caso  is  6hown 
Ir  ?lg.  'y  which  Is  self-explanatory.  is  used  to  denote  the  opticaal  feedba-ck 

gains  obtAjned  from  the  riccati  equatltjii  of  t.h«  contn.'i,  problem. 


Engineering  Implications  of  the  Form  of  the  .holutlori. 

As  In  the  filtering  problem,  the  ratio  of  the  'iisturbanco  pow;r  Q  ■*'o  the  n'-loe 
power  R  gives  an  estimate  of  the  reciprocal  tlnas  ronfH,anl  of  the.  ir;  i'';/,. 

If  the  disturbance  powjr  la  sinall  In  comparison  to  bJu-  noire  po-wer  'rnc  goto  a  rela¬ 
tively  low  gain  in  the  filtering  loop.  However,  §  m-ay  be  rs>garded  ur.  a  rough  x.i'u- 
Dure  of  how  w»]l  the  dyramlc  model  le  known,  C  being  large  for  the  cau>.'  uher-.  the 
model  dyanmicG  are  not  known  wel.l.  In  practice  usually  Q  een  lx-  aittuirij;:!.'  to  b- 
fairly  large  In  conQiarison  to  R. 

As  in  the  noise-free  regulator  problem,  tbo  time  conut#jL,al.  of  tht^  C'lntroi  loop 
can  be  ai)pro;(  t.mateiy  fape'--  i  riert  by  tue  choice  of  RQ 

Arguments  similar  to  those  In  Chapter  2  give  equivalent  time  coniitnntn  I’or  fia- 
ricenti  equations  (it)  and  (1-7)  jurd  provide  verifiable  condltlona  as  to  wl'cti  th.e 
gains  K.,  and  can  b<;  replaced  by  conulouitu. 


two 
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Gbapt-er  6 


TEE  AMPTITK  COfSTHOL  PRDBUEW 

1.  Orientation. 

It  Is  now  quite  clear  hov  Pig,  2  of  Chapter  1  is  to  be  We  asBiimF* 

that  the  equations  of  motion  are  glyen  In  form  introduced  in  Chapter  1.  We  sol-re 
the  filtering  problem  first,  yielding  the  bcpt  marked  "state  ootimator".  ®ien,  as 
discussed  In  the  preceding  chapter,  ue  form  the  "optitaal  controller"  by  opeis-tljig 
on  ^(t).  This  will  be  a  linear  operation,  represented  by  the  ssati-ix  K^Ct)- 

Assume  now  that  we  hare  a  means  of  measuring  the  values  of  the  nsatrlcaa, 

T,  0,  H,  Q,  R.  The  estimates  of  thece  partuaeterc  will  form  the  learning  states. 

IDte  other  paxa«e?t«»*8  of  the  problem,  namely  3,  U,  R  specifying  the  quadratic  per¬ 
formance  index  are  -usually  given  exactly. 

*  With  thsss  estlBUites,  vo  coc^oto  the  solutions  of  the  two  rolovnnt  riccatl 
oqmtlons  of  ths  filter  and  regulator  problem,  closing  the  "adaptive"  IcMip. 

Of  course,  nc  claim  can  bo  omde  at  this  time  that  such  a  procedure  Is  optlxiol. 

It  Is  probably  not.  Rit  the  combined  problem  of  instantant^ousiy  best  control  and  best 
estimation  of  the  stmctural  parometerB  is  too  difficult  at  present  to  be  seriounly 
studied. 


Idsal  Adaptation. 

the  valw-es  of  T,  Q,  H,  J,  ft  by  observing  the  STStCia  output  z(t).  Then  one  crnild 
design  a  controller  on  the  b&sis  of  the  theory  of  Cbapt^^r  5-  cooiblrjitlcm  of  the 

general  controller  and  this  Ideal  learning  ei'jdol  will  bs  called.  a:i  .ideal  adapt Itq 
syatem.  Ohvioueloi',  it  will  have  the  best  performance  ^ 

of  the  theory  of  the  general  control  problem  presented  in  Chapter  the  perforresnee: 
of  the  ideal  adaptive  ayatom  in  a  given  envlroniEent  can  be  determined  e3:at*tiy  —  this 
is  Just  the  general  time  varying  control  problem. 

The  comiept  of  an  Ideal  adaptive  system  has  two  major  praotloal  uses: 


(1)  IBie  e'^raluatlon  of  Tarlone  alleg<*d  "adaptive"  tieslnne,  the  determina¬ 
tion  of  whether  an  adaptive  controller  is  really  needed,  and  whether 
even  an  Ideal  adaptlvo  controller  can  do  the  Joh,, 

(2)  %e  actual  design  of  adaptive  controllers. 

Those  two  uses  vlU  be  explored  In  the  isext  sactlona. 

?=  Eyalmtloa  of  Adaptive  Designs. 

To  check  a  given  aylaptlve  5iystem  doBign,  wo  pre«»^ribe  the  evolution  of  the  ccn- 
trol  object  in  time  by  specifying  F(t),  u(t),  H(i),  5(t),  nnd  ^(t).  Wq  then 
ccapute  the  optijml  control  system  based  on  toe  lEnowlsdge  of  these  parameters.  This 

f 

gives  ufl  the  perfortaanee  of  the  optiinal  ewlaptive  eystem.  For  large  learning  times, 
a  well -designed  adaptive  system  should  approach  this  ideal. 

We  con  also  obtain  a  lower  bound  on  the  pertormance  of  art  adaptive  Bysiem.  We 
take  some  "a^rage"  value  of  FCt),  0(t),  H(t),  li(t)  and  ^(t),  or.d  design  u 

control  system  with  a  constant  control  law  based  on  these  parameters.  If  a  control 
system  is  truly  iidaptlve,  it  must  perform  better  tlrnn  one  whooe  control  lav  is  based 
on  ^the  "average"  e<iuationB  of  motion.  Of  course,  there  lo  no  guarantee  that  any 
design  with  a  constant  control  law  will  be  stable  under  the  various  conditions  wijic-b 
oBsy  he  encountered;  if  so,  this  is  a  sure  Indication  that  an  ndnptj.vr;  oyntom  in  cal.'icd 
for. 

Design  of  Adaptive  aarstens. 

A 

An  adaptive  filter  could  be  extrioioned  as  followa.  T»dte  eatlisittteB  F,  G,  H. 

P.  of  F,  G,  H,  <i,  R  supplied  by  the  learning  proceoe.  Sxibetitute  the  eotl- 
BBtes  in  the  liccatl  equation,  and  use  the  solution  of  thlB  equation  to  set  the  galnu 
of  the  optimal  filter.  See  Fig.  1,  ^rtiere  the  adaptive  adjustments  are  Indicated  by 
daabed  lines.  This  is,  of  course,  largely  an  "open  loop"  process;  it  could  be  im¬ 
proved  by  measurjjng  the  pesfomiBLnco  of  the  filter  .and  then  feeding  thlr-  inforsisiLtlo^i 
back  to  the  loamlng  procesb. 

Tf  one  would  try  1.0  extend  tJilb  soheme  to  the  control  problem  vaHilrt 

arise  eince  the  control  feedback  gai.no  arc  obtained  by  solving  the  rlccati  equation 
. backwards  In  time.  Therefoio  In  the  control  problsa  the  learning  proceae  munt  uunply 
predlctlons  of  F,  0,  H  at  least  as  far  into  the  future  ao  aeverol  tlmt  'onstonte 
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Hote  that  this  ayntom  its  completely  obeervabl-e  and  conqpletoly  controllnhlR. 
case  the  optimal  flltei  ie  deacrlbed  by  the  eqmtlons 


dXg/At  =  -  Xj_) 


which  follow  iBBWdlateJ^  from  the  rarlance  eqiaatlons  in  Chapter 

Ifow  suppose  that  and  r^j  are  the  only  unknown  pammetorB.  Ilflaen  It 

follows  from  (5*7)  that  If  ^  estimated  then  (5.7)  could  be  solved 

to  set  the  j^ina  tn  (5»5)  and  hence  achieve  adaptive  behavior. 

The  variable  €  «=  Kj^(t)  -  ic^(tlt)  =  x^(tlt)  +  T(t)  is  idi iir -^olBe ,  has 

flat  spectrum  when  the  system  (5.5)  !■  optlmel.  When  (5.5)  1=  not  optimal  e  (with 
X.  being  the  es'^^joate  of  varies  as  In  Fig.  2.  One  can  compute  the  area 

un-^ar  tbo  specti^ia  .  r  f.  from  0  to  oo^  by  passing  e  throxigb  a  low-pass  filter 
and  then  rectifying  it.  Likewise^  the  area  under  the  Bi>ectram  of  c  from  to 

Sto^  is  computed  by  means  of  passing  c  through  a  band-t^aaa  filter  and  then  rectify¬ 
ing  It.  Therefore  a  convenient  learning  process  Is  provided  by  the  nonlinear 
differential  equation 

(5.8)  dX,/dt  «  kjb  k  lr^^p,  ^  P 


where 


I«|  _  »  result  of  passing  c  thisjugh  a  lov-paas  filter  and  rectlQrlngj 
jej-  B  result  of  passing  «  through  a  hand -pass  filter  and  rectifying) 

'  Da  if  t 

!Ebe  final  adaptive  system  shorim  In  Pig.  5  Is  doscrihed  by  the  eqtmtlonB  (^.3), 

(5.7),  (5.8). 

Ihls  system  can  be  made  more  sophlsi^loatad  by  detoxmining  tbs  constants  a,  B. 

and  It-  so  as  to  prorlde  the  videat  stability  margins,  eirst  je-Bsing  e  throuith  an 
5 

exact  copy  of  the  filter  loop. 

In  [Buey,  19593  a  rather  detailed  system  is  described,  and  re8\ilt8  of  coogiuter 
simulation  are  given  vhlch  substantiate  the  theoretical  analysis. 
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RECTIFIER 


Chapter  7. 


OUIDIHO  PRIBCIPLES  OF  MOMERICAL  COMPUTATION 

One  of  the  inyortsnt  objecttres  of  thle  project  is  to  develop  taethods 

for  dealing  vlth  cewiplex  control  systeciiB.  Since  eDophasiH  vblb  not  on  the  auaalyslB 

but  on  the  opt Inal  bynthesls  of  these  systens,  analog  conpxxtat  1  on  'wud  of  the 

question.  Prom  the  beginning  we  were  striving  to  obtain  efficient  lacthodr  of  dlgi- 

•oeil  cooputatlons .  As  a  matter  of  fact,  the  methods  pr«»sented  in  this  report  would 

be  rather  aidnmjrd  to  apply  without  the  use  of  a  digital  computer.  ’Rilo  is  the  prxco 

that  must  be  paid  to  obtain  methods  which  are  applicable  regardless  of  complexity  of 

the  problem.  Only  hy  a  cooblnatlon  of  Imaginative  coeaputer  utill  wtion  and  advanced 
•  «  ' 

matheiBatical  techniques  con  engineering  problems  of  complex  system  design  Ixi  effec¬ 
tively  attacked. 

tiulded  by  this  philosophy  of  approach,  we  developed  a  general  computational 
froneworJe  fOr  problems  In  systems  theory.  The  following  specific  objectives  have 
been  accoaq>llshed. 

(a)  All  cooputatlona  should  take  place  in  the  tlae  dosaaln.  Ihio  vclb  necessi¬ 
tated  by  the  nature  of  the  underlying  mathematical,  theory. 

(B)  The  computations  should  be  "eigenvalue less".  That  is  to  say,  no  inversion 

of  laplAce  transforms,  solution  of  hlgh-order  algebraic  equations,  etc.  should  be 

# 

i-o<iM.ireu..  The  methods  we  are  us'ng  work  easily  for  l!/-th  order  (whlcb  ip 

the  waxiiruin  size  for  which  they  have  been  designed)  and  con  surely  be  extended  to  at 
least  30~th  order  systems  wi.thout  the  need  for  basically  different  numerical  rantlioda. 


Thoue  approaches  run  Into  serious  numerical  dlfflcviltles  idien  tte  order  of  the  sys¬ 
tem  exceeds  perhaps  10;  the  difficulties  becons  probably  fatal  when  tho  order  exceeds 

50. 
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( c)  The  Bpeclflcatlona  of  toe  aygtea  ghould  be  given  in  matrix  fom.  Thle 
18  zieceaaary  for  eaee  of  proerreaning  and  desirable  In  order  that  the  progi^aipfl  have 
zaaxlnum  flexibility* 

(D)  8«uOled»dftta  gyetews  shovild  be  possible  to  ty\iat  vlthout  speciaa  t.echniquea* 
This  le  really  a  by-product  of  the  oatheaBatical  theory.  Iftich  of  the  elaborate  engi¬ 
neering  theory  on  flaB5)led-uat».  syatems  can  be  dispensed  v^th.  The  aame  prognurus  can 
be  used.  The  principles  of  solution  of  specific  probleaw  in  the  continuouE  and 
saapled-data  cases  differ  only  in  ninor  details. 

(K)  The  results  should  be  displayed  in  a  neat  fom.  We  have  adopted  standard 
formats  for  prlntlztg  aatrices.  Every  effort  bns  been  made  to  present  result;  b  of  com- 
pvtations  in  a  clean,  u^able^  and  co«opJ;ete  fora.  Ve  are  nenrlng  the  ntage  where  the 
end-result  of  a  specific  system  oxitlmlzation  problem  le  a  "book"  prod>iced  by  the 
coatputcr,  which  describes  the  problem,  exhibits  the  answers,  provides  partial  checks 
in  the  C4MLrse  of  the  ccnputatlon,  etc. 

The  principal  program  from  the  numerical  point  of  view  its  n.  auhroutine  for  the 
ccmQputatlon  of  the  exponevitlal  of  a  matrix.  This  In  the  central  part  of  tranBLer.'. 
computations.  One  can  also  use  this  subroutine  to  solve  efficiently  ucsss  prob.letsjr. 
which  at  first  eight  rcqxiire  only  elamentsxy,  octhodo. 

Consider,  for  instance,  the  linear  matrix  equation 

(l)  F’P  >  PF  -  -  Q 

vbere  Q  and  F  are  glveu  constant  n  x  n  matrices  and  one  wants  to  find  a  syy.- 

metrlc  n  X  a  matrix  P  sUlsfylDg  the  equation.  TuIb  probleaa  occurs  In  the  second 

method  of  lorapuncr.  In  constrxicting  a  lyapunor  functJoni  for  a  linear  nyste^i  vith  con- 

stant  sflEfflcients.  of  cenrse  (l)  Ic  Just  a  set  of  n(n  +  Ij/s  eqviatlonB  in  the 

n(n  +  l)/2  unknown  elcawnts  p^,  ...,  p^j  ...,  p,,^;  ...  p^  of  the  oyroK-tric 

patrix  ?.  Heoce  the  problem  can  bo  solved  usina  a  standard  matrix  inversion  iiub- 

rcuiine. 

There  are  two  difficulties  with  this  method.  PIrat,  if  n  *  10,  then 
“  f5,  ■!quatlona  ^ 

tttclxAiquMm  Second^  the  data  cot  arr^jaged  in  euch  a  vny  that  the  mtrlx 
deflnisg  the  n(n  +  l)/2  Unesjr  equations  In  the  elements  of  P  can  be  read  off  by 
Insxmctlon.  As  a  matter  of  fact,  exceedingly  tellous  bookke^vping  1b  seeded  to  bhtaln 
this  matrix,  slxkce  It  has  (55)^  "  5,025  elements. 


CkM  can  coaQMte  P  by  a  dlff«2«nt  iMthod,  hovarrar.  It  la  v«Il  )cr9im  that  if 
F  la  a  Btabl«  laitrlx^  then 


o 

and  this  expreaaion  can  be  readily  evaluated  by  using  the  riccftti-equction  tubrjsatirvs 
(which,  in  tunv;  la  baaed  alaoet  entirely  on*  the  exponential  eubrcntijMs) .  In  fact, 
consider 


Kt)  -  f 

t 

Differentiating  with  respect  to  t,  we  sec  that  P(t)  sctlsfien  the  differential 
« 

equation 

(2)  -P  -  F'P  +  PF  +  Q 

which,  is  a  special  case  of  the-  rlccatl  equation.  If  all  eigenvalues  of  F  bn 
tive  real  i>art8,  then  as  t  -  oo,  P  -♦  0.  If  P  =  0,  then  (2)  r«;duccfi  to  (l). 

The  solutions  of  differential  equation  (2)  can  be  ccroputed  rapidly  and  accureite.ly 
by  means  of  the  rlccatl-equatlon  subroutine  which  In,  In  effect,  a  apeclal  iteration 
procedure. 

Thus  ve  see  that  by  reducing  a  tri'/ial  algebraic  proWem  (l)  to  a  nontrivial 
analvtlc  one  (2),  a  great  deal  can  be  gained  frcni  the  point  nf  view  of  efficiency 
and  ease  of  numerical  computation. 

The  following  tour  cnajiterB  contain  a  ucoe^  ipuion  of  the  main  suu..  wt  i -.ju  wiiicii 
were  developed  to  date.  Itach  jrabrcutlnc  contaluB  ace&e  contribut.'.on  tc  nuiaericii-I 
analyslB.  To  aid  the  eventual  users  fatr.l.y  detailed  explanations  aT-o  given  concern¬ 
ing  the  origin  of  the  subroutine,  the  methoda  of  comptitatlon,  and  nyjT«erlcal  checka. 
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Chiipter  8 
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♦(t,  t)  »  oxp  0*P  «  1, 


vhloh  is  trivial;  aeccnd,  ve  nsuat  thotr  thax  "xpt(i  -  Xp)i  i  natlsrieB  the  dllfercn- 
tlol  equation  'l. 3).  By  the  definition  of  tha  derivative; 

(t  +  h  -  t^)F  (t  -  t  )F 
o  o 

d{e3q)(t  -  t  )F])/dt  =  Ilia  - — - r - — - . 

°  h  -»0  ° 


tP  and  hP  cooiamte. 


(t  -  t  +  h)F  (t  -  t  )F  .  „ 

'  o  '  o'^  hr. 

e  me  e 


Hence 


(t  -  t  )F  ^  ^  r 

de  ^  /dt  =>  11m  - — r— —  '  e 

h  -.0  ^ 


By  (1.1); 


=  F  •  e 


(t  -  t^)F 


t  )Y 


■which  voe  to  be  proved.  Note  that  this  proof  fallB  In  general  if  F  /  consi^uit.. 

This  nay  be  demonstrated  also  by  tenwlse  differentiation,  (as  we  barre  “ireadj 
shown)  because  the  8erle8(l.l)  converges  unifonaly  on  every  interval  lO,  tj. 

Soae  other  facts  which  nay  be  proved  about  arc  (oee  [Coddin^ton  and 

Levinson;  1955) 


(l.M 

(1.6) 

(1.7) 


lle^li  *  (n  -  1)  + 


where  n  1b  the  order  of  the  cantrl?.. 


A  -t  E  IB 

e  »  e  e  if  and  only  if  A  und  E  ccraHute. 

J’^FJ 

e  u  J  e  J. 

Deterainant  e'*'  j.  gtrace  1* 


xF 

•laie  last  tiKiorea  snows  that  e  is  always  nonojngular,  Koir.'eover;  tlie  coluisiia 
of  etre  n  linearly  independent  aol’utlons  of  (J.p).  Thus  oixy  aolutlOD  (‘f  (1-^^) 

can  be  obtained  by  a  linear  combination  of  the  coluns  voctora  cf  . 

2.  Progr«/n  Alg^iths. 
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For  cofipvttlng 
serlea  (l.l)  is  used 


tP 

9 


the  sum  of  at  moat  the  first  thirty-six  terms  of  its  defining 
UtMS  we  eoapute 


(2.1) 


?  tV  ~  tP 

^  j  t  "  ®  • 

l»o  • 


Ihe  sm  (2.1)  la  actually  cojaputeu.  as  follovs.  Let  be  the  ith  term  ot"'  the  expnn- 

sion:  T  «  I,  T,  »  Ft,  ate.  The  sua  is  eccunmlated  and  T.  ,  in  obtained  from 

o  '  1  '  1+1  1 

by  multlpllcatloa  by 


1+1* 


The  following  tBOtlrates  why  xhlrty-alx  term®  ore  used  In  (2.1),  and  Rivee  a  ccn- 

diticn  lasder  which  the  result  can  be  expected  to  be  accurate. 

In  the  UM  709  a!"«d  7090  a  little  more  than  eight  significant  digits  ojpc  carried 

when  operating  In  the  single -precislcn,  floating  point  mode.  This  lopoacs  llmltatlone 

on  the  accuracy  of  the  program.  Consider  a  simple  cosine  series.  We  know  that  for 

any  rol’JS  of  the  argument  the  absolute  value  of  the  fiinctlon  Is  one  or  lesc.  Yet  if 

20 

the  argument  were  20,  the  term  20  i»  so  large  that  the  addition  to  It  of  n  number 

of  the  order  of  1  ija  magnitude  does  not  effect  it.  That  la,  if  any  t  ..m  of  n  series 
exceeds  10^,  we  know  that  no  answer  of  the  order  of  1  or  less  in  magnitude  will  In 

general  be  correct.  Thus  If  we  want  an  answer  that  Is  correct  cc  foui-  dec IrTt-'i-l  r  ’actes, 

k  t 

no  component  of  the  sum  may  exceed  10  .  The  larscst  term  of  the  c  ocries  it; 

1  '' 

where  J  is  the  smallest  Integer  such  that  - — —  <  1;  fr^trefore  T  ~  where 

u  +  +  .  ,  .1 

**  t/ 

J  =  it],  tho  greatest  integer  less  than  t.  For  otir  puri)Oo«:«  — ,  should  always 

be  lens  than  IG  ,  which  that  t  sho\jJd  be  less  tlian  JO;  since  -  Ji?  . 

ic: 


Without  etteraptlng  to  discuas  the  problem  more  rigorously.  It  was  decided  that. 
iiP|l*  It  j  <  10,  then  the  answers  could  be  depended  upon  to  four  decimal  pl'iccii. 

~  10^  • 

fuz-Miur,  -  iU  it  la  <5Vidw,t  that  uo  more  than  thirty  nix  terms 

nastd  be  carried.  On  the  other  hand,  if  lo  greater  -han  10  then  frr  wro 

^  5p.  W: 

greater  than  10  ,  and  the  validity  of  the  answer  is  open  to  question  djiyway.  Ihi- 
fortunataly,  there  ic  no  error  return  if  such  a  condition  occurs.  Sjtoi’  return 
occiirs  when  one  of  the  is  greater  than  10^  (machine  overflow);  cooputation 

of  (2.1)  stops  when  a  terra  la  loss  than  10~^  (underflow). 


:  r 


3,  (flecks 


(A)  vas  coi^uted  for  the  15  x  15  nilpotent  oatrljc 


°li.  =^14 

0  0‘j^ 


(vhere  is  the  l4-dlnenBional 

zero  column  vector  and  I,,  le 

14 

the  14  X  l4  Indent Ity  matrix) 


for  t  r,  .1,  1.,  5.,  and  10.  For  all  thes^;  values  of  t,  this  checked  to  six  signi¬ 
ficant  figures  aaid  approximately  thirteen  dec.'''«al  places. 

(B)  Choosing  at  •’randea"  a  15x15  cs&trlx  F  such  that  HFjj  »  91  and 

tA 

IT  ' 

A  .  ^  O  P  P 

t  a  0.10989016,  e  ^  and  (((e  )  )  )  )  were  computed  and  the  results  printed  to 

tA 

^  p  p  o 

ei^t  significant  flgureB.  Tt  wac  felt  that  (((e  )  )  )^)  wub  probably  quite  riose 

tA 

to  the  tixact  value  of  c  .  Comparison  shoved  the  tvo  resuits  to  be  the  same  to 
about  six  decimal  places,  vith  the  smaller  elements  losing  acc’jracy,  being  correct 
to  only  four  or  five  significant  digits. 

*  For  the  matrices  Involved  in  this  check  see  Flguixt  1. 


(C)  The  exponential  vas  computed  for  the  7x7  diagonn.1  ruatrLx 
dlag(-10,  -4,  -1,  0,  1,  4,  10)  vith  t  ■«  1.  The  result  Is  In  Fig.  2. 

Because  this  matrix  is  diagonal,  the  scalar  analysis  used  In  Sect.  ?  applied 
exactly  and  the  answers  accord  with  this  very  well.  In  the  submatrix  (~10)  where 
we  were  not  only  at  the  limit  of  the  acceptable  range  of  application  nut  were  taking 
differences,  we  barely  msnaged  to  have  accuracy  in  four  decimal  piaces.  In  fact  If 
the  enswer  bad  been  printed  in  the  four-place  rounded  format  which  we  u«e,  rounding 
would  have  produced  the  wrong  answer.  However  ,  where  differences  were  not  bcliiK 
taken,  as  in  the  submatrix  (].0),  the  answer  is  correct  to  seven  significant  fJgures, 
which  again  we  expect.  The  same  results  ax's  true,  vith  dtrcreacing  signiftconcr;^-:! 
the  Bubmatricec  (4)  and  (-4)  and  (l)  and  (-1). 

In  general  however,  the  accuracy  seems  adequate  for  our  purposes. 


4.  References. 

E.  A.  CODDINGTON  and  N.  LHVINSON  (1955)  "Theory  of  ordinary  differeTitial  equa¬ 
tions,  (book) ",  McGraw-Hill,  1955* 


«c*privt  svsUNS 


86 


TfCTEtlTTr  coicuks  is 


Of 

o 

^  ' 
^  I 

“i 


^ ; 
a  p 

ar  • 


ksbsbsbs 


;!j 

a; 


X  , 


I 

I 

I 


I 

( 


1 


87 


fPiURE  i-B.  D^(mmmmmWA\\m}^r. 


^38 


fiGU.R£lt£;_ 


Numircr  ftcws'15'  ~  KUfir(incr-cecv 


i  ! 

I  i 


i 

I 


f 


— _  imf  ««*  «■«  fw*  lij 


Ul  W  4wlu|  UifiAi  UJ 


POlrtiftRg  O  e>p  Ji  ff^  tf 
1  rn  f*  U|  tn  —  <  l«  N- 
If.  N.p  »  *  O 
I  •«  N  '*»p  ^ 


(DO 
I 


>^MiO  - 

)0»PDOOOB9 
I  [T  1  I  J  T  t  I  i 
liUUJill  MJUI  AlUl  wuui 

%N  M  »1  P  >  ^• 

rr^p  BLiu  *»A>  — 

»0*  -  <>^U;w'. 

NO  /)«-*Ow^l>0 

00^<Nh*^-^ 
‘0m04D0  NCO  /i*^Inu^)0<DM^ 

^fevi4VOO 

f^l^  —  0*-F-^-r4»nK> 

»0|0000  30DOO.O 
Ml  I'M  » 


»  «  r>  « |0  ^- 


o^o  w 

t“  ojo  in 

K  •-  p  *n  ^  *0 

^  o  in  IP 

P 


‘  o  in  < 

>00000 


>in%n^' 

>  o  o  o  < 


><c<^4imoo  R)<or- 
’OOOOO  oopooo 
I  I  I  I  I  I  it  I  I  I 
U^UJUIW^MUI  AlUiHU/puJ 
tn^«l^C^N9«n  o^por^** 
I'OjrOp^  s.op>Ojrin 
oO^<rpo 
u^Hinou 
u  cj  p  •»  rv 
m  M  o  ^  «o 

^  p  •«  {n  jf 


_ _  oopopopo 

y  II  I  <1  Ml  I  i  I  [i  }  il  t  [i  I 
UiUJWUlUlUiUlWW^tJJMUl 


mi-o  c^ntn 

Y  o  p  o  ,o  o  |o  o 

ui  ;u>  Urf 


»  'O 

-  -  _  ,  -t  o 

(  i  I  t  I 

/ 1.1  u;  u;  c<j  UJ 

K>fN^^lf>  — 

p  •nSn  m m 
^  'O  o  o  »  — 

)N  ^  V-  m  >0  o  -c 

f“  — 

ri  —  <N  u  «v  o  fe  »n  ri  ui  in  rw 

BU  o  p  in  L>  In  ••'j  H  nj 

*•  *i#  *  •  t  ■  9  • 

IU>  o  P  C’  O  O  p  Ij 

it  t  7  I  I  ii  » 


P  >0 

p  o  p  u  P  o 
j  !  :i  M|  t 
p  UJ  UJ  ill  p  iu 
&  o  l»  m  pi  CM 
p  O  P  nj  l>  ^n 
fcn  —  o  CM  tn  >0 
m  p  o 


■.ec 

^•5 

!0 

P 

1 

UJ 

fe? 

.0 

«>  ; 

.K 

>  lA  n- 

lopopo 

t  fl  t  •  I 
j  lu  Uj  lU  p  ui 
•n  t-»  cn  o 

•  —  No 
-  o  N  o 
o.- 
»  ^1  In  o  ft  o 
I  #  b  »  ^  « 
j  o  P-  jr  m  o 
■  ^  pj  ^  flO 


fooKflOp-^'-f-ofcMn 
uooobobopo  :)obooo 

I  I  ji  j  !*,  I  I  r  I  |i  I 

•  «  ‘  I  I 


p  p  ^  ) 

P  O  )C'  O 


I 


-^t  <}  *0  ^  fm  90 

000000 

.1  «  I  t  I  I 

UJ  UJ  Uj  t  •  UJ  UJ 

N 
in 
rv 

rw 

p  —  00  M  p  c^ 
N  00  jr  w  »n 


*00000) 

!t  t 


I  I  i ♦  Ml  » [»  » 1*  t  n 

UJ  UJ  U>  u.<  p  w  L.’  us  lu  UJ  p'l 

n-  1%  o  o  p-  oj  ^  j»  L'l  . 

»  *>—  -A  L’>  *<1  p  ••*  «o  w>  r'  *  ' 

ij  f\)  o  p  o  Lr  c»-  fn  UJ  ^  -tv ; 

ruf^  o' 

•j  t.  o  jr  |[T)  r^.  p-  o  1 

4f)— jr  —  •-fco«“'U»ini 

•  {•  *,*  ».'•  •■*  ol*  «• 

•»  |o  o  lc>  «>  b  o  b  v>  h-s  w 


‘  •  ’  i  I 

k>H}k)*okj«€boioo^’0'0'Cs70'0*o*o-o«o<JOinuio^*o*otn 
popupopopoooooooooo^jooooooootoo 
n  I  fl  I  M  I  4  I  [f  t  *1  I  •  •  t  I  »  1  i  I  I  Ml  t  *  j  I  I  ;»  I 
UJ  u;  U;  UJ  UJ  ui  Ui  lu  p  u*  ill  U»  u*  \..  uj  u-  Uj  wj  uj  lu  ju.  u.  w  uj  uj  u'  u.  uj  W.  w 

Pr^tn  —  uimwtfaijr  —  —  o« 

to  *0  K  m  K>  FO  K.  .0  P  j»  to  tn  3>  u>  in  o  cv  rj  t>  -o  o  Q  <-j  AA  ^vj 


op-fMNJWp^p 


3>  u>  «n  o  jr  CK  jr  <M  o-  o  -o  o  <u  An  ' 

fo*  —s  tlj  U.‘  .»  m  W  O*  WN*  r«- 

.»  r.  >  o  h-  ru  w'l  JT  o>  o  r-  —  o  vn  JT  <.j 

%rt  ^  or  fv  V  V>i  >0  .^O^Ot^CN^-  —  INOJCJCJOO*—  <Ni<M 

in  —  klln^onJfM^-  »o•A^sJlnoot^^c►o•  ji  v—f^o 
^  *'*  ^*  -'**-““““*“'*'*  rj  pn  "f-i  p  ^  o  —  —  Gj  -o  'n  ,-■• 


Ip  o  p  o 

I  ll  I 


5 


1 

;(-> 


i 


i 


1 

I 

I 

j 

1 

I 


69 


Corqr.ited  Value  o!.’  Expcmentlal 


91 
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Correct  Ya':u2  of  Mitrlx  iiqpaiential 


Chapter  9 
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Bie  Bolution  to  (1.2)  for  x(0)  ■  0  may  be  written  as 


v(t)  “  ^'^0  u(t)dT. 

o 

To  obtain  a  conqplete  syetea  of  solutions  when  u  is  constant,  replace  the  vector 
u  by  the  identity  matrix.  Then 


r{t)  n.  f  ■  '^^^OdT. 


Making  the  substitution  t’»  t  -  T,  the  integral  asoiamse  the  nLi^ler  foi-m: 


T  *  IT 

r(t)  =  f^e'  *dT*0 


Thin  Is  the  matrix  vrtiich  the  subroutine  "integral  Exponential’'  computes. 

2.  Program  Algol Ithm. 

The  terms  T,  of  the  defining  nuttrix  are  obtained  as  they  arc  for  the  mahrlJi 
*  ^ 
exponential 


T 


1+1 


T 


tF 


1  1-^1 


; 


however,  T  ^  tl,  not  1. 
o  ' 

Hke  ai’gumcnta  concei'ning  permiBsible  range  of  t  Kiven  in  Chapter  B, 
are  applicable  here  also. 

I,  C-hecKs . 

a)  "he  integral  exponential  was  compviteol  for  in  x  in  r.llpoteut  1. r t>: 


0 


O' 


11 


(where  lo  the  il-d  onal  ^ero  column  vocLor  and  In  the;  i-'i  x  i !.de:i- 

tlty  rtwitrix)  for  t  -  ,  I.,  !.,  arid  10-  For  ai  .1  values  of  t,  this  checked  to 

six  significant  figures  and  approximately  thirteen  deciinal  places. 
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Chapter  10 


•USE  TRANSIENT  PRCXIRAM 


1.  Theory. 

The  TranBient  Program  is  designed  to  give  a  time  history  of  the  state  vector 
y  of  the  system; 


(1.1) 


i  =  Fx  +  Qu 
u  =  Jr{t)  -  Kx 
y  -  Hx. 


problem,  as  explAlned  in  the  first  Sections  of  Chapters  8  and  9  has  the 

solution: 


(1.2) 


x(t  t)  =  r{t)clt. 

°  t 


under  the  assumption  that  r  Is  a  constant  vector,  the  solution  (1.2)  may 
be  written  stepwise  as  follows: 

x(t  +  T)  .  -/f'-  “>x{t  )  .  si  ^  ™)axGJr 

o  '  O  ■'0 


where  T  is  the  sampling  period. 

Thlfl  enables  us^  ss  1.-:  the  Rlccati  Program,  to  obtain  a  step-wise  oampllng 
of  the  anaiyxic  solution  of  the  pjroblem,  after  computing  the  exponential  and 
integral,  exponential,  of  T(F  -  GK)  only  once. 

Program  Algorithm. 

‘Iho  equations  which  have  been  laechiuiiaed  are  the  .rcllowing; 


x(t  +  T)  a  9x{t)  +  pu 
u  *•  (Vr  -  £x 
y  »>  H*. 


lAMre  th®  matria«a  r>  J>  A,  E  wo  Inpvrbs,  as  well  as  T,  K,  tiie 

tnaxlmum  ntuciber  of  steps,  and  anottaor  saaqpllng  period  t  «  Tk,  k  an  integer,  the 
use  of  which  vili  ho  oscplainod  beloi?. 

Input  might  ho,  for  instance  to  Bolre  the  aystoa  (l.l) 

r  -  ;V(''  -  ®>dTO 

0 

A  ■  J,  E  ■  0. 

f 

It  is  apparent  that  If  r  is  a  constant  and  the  inputs  arc  as  they  appear  ahcnre 
then  u  a  Jr,  a  constant  rector,  and  the  equation  for  u  is  superfluous  in  the 
stepwise  procedure. 

To  accommodate  the  case  when  r  is  a  piecewise  constant  (sanpled)  function  of 
time,  the  second,  t,  sampling  period  has  he^n  provided.  Every  k  steps  of  length 
T  in  the  ccoq)utatl,on  of  x,  the  value  of  r  is  usei-l  to  compute  a  new  value  of  u. 
If  r  is  a  constant  this  offers  a  saving  in  machine  time  since  choice  of  a  k 
greater  than  N  will  prevent  reference  to  the  second  equation  except  at  the  begin¬ 
ning  of  the  run.  In  this  way,  it  is  easy  to  find  the  response  of  a  saapled-data 
system  between  sampling  points.  As  in  the  Rlccati  Progreun,  observe  that  this  lo  not 
a  stepwise  Integration  procedure;  the  only  errors  ajce  round-off  in  the  computer  and 
whatever  is  Involved  in  assumiiig  r  to  be  sufficiently  weU.  approximated  by  a  step 
^mctiCii. 

Another  feature  of  the  program  is  that  not  the  state  variables  but  linear  com¬ 
binations  thereof,  are  printed.  In  several  problem?  dilch  beve  beer.  conKjclo-rv-d. 
e.g. ,  one  concerr’lng  aa^cllite  or'' ontatlon,  the  state  variables  could  not  he 
but  oui^  linear  coobjnetlonD  of  the  state  variables.  To  be-  able  to  compare  raichinc 
results  with  exi)erjinental  data,  the  Transient  Ibrogram  has  on  additional  inj^ut  matrix 
H  and  the  vector  of  obsejvables  y  =  Hx  is  printed. 

At  every  interval  T  the  complete  output  io  printed.  This  consists  of  the  tim;; 
two  conqjonents  of  the  r  vector,  seven  conq^onents  of  y,  and  three  cociponeni.s  <:n  u 
The  problem  is  terminated  by  exceeding  the  maximum  nxunber  of  steps  N.  of 

programmed  space  limitations  N  must  ijc  loss  than  200, 


Ctwipter  11 


SSE  HA3SIX  RICCATI  EQJATTOS 


afi£SZ* 

A  system  of  2n  liziaar  differential  equations  is  said  to  \>e  honiltonian  (or 
oaaoaical)  if  it  can  be  vrltton  In  the  forwj 


(1.1) 


4Xj/4t 


dc. /dt 

-  V 


1  =.  1. 


n. 


Vhsre  Pf  ,  the  hamlltonlan,  is  a  haaogeneoufi  quadi-atic  pclynooiial  in  tlic  y,  nnn 

vlth  coefficient a  vhich  are  functions  of  time. 

The  most  general  such  function  nay  be  written  ae 

S 

(1.2)  2^-  [x,  y]H 

> 

where  H  la  a  symmetric  matrix,  n  can  be  partitioned  as 

A  C 

•'l.t'.  H  = 

V*-**  /'  /  — 

C'  -B 


where  A  and  B  are  symmetric^  C  is  arbltmryj  the  negative  elgn  beforvj  B  lo 
pure'!”  for  r-v.r  later  convenl.ence,  AJll  these  is, trices  my  he  fiaictlony  of 
Substituting  {.1.2-,/}  into  (l-l),  we  get 


(1.4) 


mia  equation  posseesea  an  Inagportant  kind  of  syninetry  which  we  can  ntar« 
follows; 
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(1,5)  THEOREM,  A  2n  order  eyatem 


± 

ir 


-  z 


y 


is  hami  Itonlan  if  and  oriy  if 


Z  “  JZ  *  li 


yfaere 


-t  ■» 


0  -I 


0 


Proof.  Only  if  I  TSiln  followa  from  the  fact  that  the  2n  order  3yr,tem  (l.U) 
^bove  —  vhlch  Mae  derived  from  the  most  gencraJ.  posclble  liamaJltonlan  f\uactlon 
—  satlsfleB  the  condition. 

If:  Consider 


C 

-A 


-B 

D 


where  A,  C,  D  ore  arbitrary.  Then 


JA'jr 


./Z'J  - 


- 

— 

"1 

-K> 

C’ 

B 

c  ! 

D' 

-B’ 

D 

E 

1— 

■ 

I'-D’ 

-B’  " 

~C 

-B"! 

1 

MV 

-A* 

-A 

D 

L 

.. 

J 

I  =  n  X  n  .Identity  matrdjx  and  0  »  n  x  n  y.ei’o  naxtrix. 
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and  A  e  A’,  S  ■  B',  and  D  »  ~  C’.  Tjut  tfcee©  are  ;3\ust  the  conditions  satisfind 
"by  the  matrix  Z  in-(l.U.)  above,  and  it  la  apparent  that  the  form  of  Z  determines 
that  the  system  is  hamiltonlan. 


(1,6)  COROLLARY,  is  an  eigenvalue  of  Z  so  Is 


Proof.  Zx  r  X^x  in^lies  JZ’Jbc  =  X^x  and,  since  J 


-1 


VS 


•  *jbc  a  — X^JjC. 


(l.T)  THEOREM.  Let 


0(t,  t^)  = 


e^iCt,  t^)  e^gCt,  t^) 


(1.8) 


'•)  8i2(V 

*•0^ 

’-o) 

^e-J*o’  ®g£do> 

m 

Proof.  If  6(t,  t^)  Is  the  tranBltior.  aaatrix  of  jt  »  2x,  then,  ae  io  - 

known,  the  transition  matrix  of  y  a  Z'y  is  e‘ (t  ,  t).  note  t.hc.t  J'  :t  -j. 

o 

By  Theoreo!  (I.5) 


Z’  - 


Tnjt  Jy  «  X.  Thisn  ^  x,  jy  ..  JS'y  ^  ~JZ»J  x 
step  fclloving  by  •OMsoinain  Hence 


•.TZM’x  -  JZ*Jx  Zu,  thr  3furl 


J'0'{t  .  t)j  =*  0(t,  t  ) 
o  o 

e(t^,  t)  -  ,7B’(t,  t^)j' 


la  identical  with  (1.8).  Q.  B.  D. 
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103. 


Chootlng  P(t^)  to  S7»aetriQ  guazwataeB  that  P  Is  flyumetrlc  tuod  then 
P(t)  l8  alvaara  synnetrlc.  Althou^  formula  (1.9)  does  not  iieea.  to  be  symawtric,  it 
H.  as  can  ba  Tarifled  u«ing  relations  (1.8). 

la  the  special  eaae  vhersj  the  rlccatl  equation  la  linear  (B  =>  o)s 


®12  *  ‘*12- 


But  ®  bocaaxse  9(t^,  t^)  «  1,  therefore  012  “0. 

.  -1 

But  from  equation  (1.8)  ve  ^ee  that  9-,^  =  0^.  •vnd 


(1.12) 


P(t)  ”  fOoiCtp  t^)  +  e2g(t,  t^)P(t^) t^)  If  B  =  C. 


Biua  ve  haye  proved: 


(1.15)  THEOREM.  5^  solution  of  the  rlccatl  equation  (l .  1. 1 )  mjiv  xpr^:BBed 
in  terms  of  the  transition  matrix  of  the  haalltonlan  system  (l.H)  If  B  0,  the  aolu- 
tipn  alvays.exiatB  and  le  given  by  (l.l2).  ^  B  /  0,  the  eo3.utiCTi  U  Klvcn  by 
(1.9),  ih  an  Interval  of  time  vfaere  t^)  +  lr>  nonslnKolar. 

last  condition  rules  out  the  ao-caliod  "conjugc-tc  pointG"  of  the  calculus 
of  vnrlationB.  An  example  where  a  conjvigate  point  does  occur  was  given  in  &:ct.  7 
of  Chapter  2. 


In  case  of  the  rlccatl  equation  (Ji»^.6)  of  Chapter  2,  the  matrix  Z  hao  the 

form 


(i.iM 


z 


F  -OR'^G’ 
-H'QR  F« 


In  the  case  of  the  rlccatl  equation  (3.3)  of  Chapter  14-,  the  laatrlx  Z  liaa  the  fonn 
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(1.15) 


z 


*-F' 

-005’ 


F 


2.  F3*GgrRa  Alogyitlam, 

The  tras'.sition  matrix  of  (l*k)  can  be  computed  easily-  only  vhen  Z  Is  a  con¬ 
stant  matrix.  In  tost  case, 

(2,1)  e(t,  t^)  «  exp[(t  -  t^)Z]. 

f 

Hie  progrca  first  computes  (2.1)  and  then  P(t),  by  subRtltutjjifl  the  four  cub- 
matrices  of  0  Into  (1.9)  or  (1.2).  In  this  vay  one  obtains  a  step-wise  solution 
of  the  riccatl  equation  without  any  truncation  errors,  subject  only  to  roxmdoff 
errors  In  computing  0  by  means  of  iihe  exponential  Bubrou'’:iiie  of  Chapter  8. 

At  each  step  P(t)  is  symmetrised  before  proceeding  to  the  next  step  by 
replacing  it  by 


2 

Symmetrlzatlon  is  absolutely  osaential  because  otherwise  uncantralJnble  roundoff 
errors  may  accumulate  in  the  antisyumetric  part  of  P(t). 

Hie  input  to  the  program  consists  of  the  matrices  A,  B,  C  (C  arbitrary, 

A,  B  symmetric),  a  syrametrlc  matrix  i*(t^),  a  sampling  period  T  at  which  in- 
terrals  the  matrix  P  will  be  computed,  a  convergence  criterion  number  e,  a 
maximum  number  of  Intervals  N,  and  yario\ia  printing  cod.e8. 

The  "sampling  period*'  T  ■  t  -  t^  for  the  riccati  equation  may  be  fa'bltrTu-  ', 
subject  only  to  the  restriction  tijs.t 


which  is  necessary  for  +he  converjsenco  of  the  exponential,  subrout l(ve  (tee  Chapter  8). 
Since  only  the  ratio  of  Q  to  R  matters  in  (l.lH)  or  (1,15),  cme  shoull  state 
these  quantities  so  that  OR  ^G'  and  H’QB  nre  appraximately  of  tlua  satno  order  of 
magnitude  —  this  will  keep  jjA|]  small. 
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IHiis  problem  is  terminated  in  one  of  two  waye.  Either  the  tBaxiraum  number 
of  steps  Is  exceeded  or  the  convergence  criterion  la  aatisfied. 

The  convergence  criterion  is  that 

b€'  less  than  c,  an  Input  number. 

When  p(t)  is  coiqputed,  K{t)  ■  R“^G’P(t)'  pr  K’(t)  »  P(t)OR"l  can  be  alao 
coa^'utwd  and  printed,  if  desired. 

Print  controls  enable  the  customer  to  print  K(t)  nnd/or  P(t)  at  every  step, 
every  fifth  step,  only  at  the  final  step,  or  never. 

As  stated  above,  the  Rlccnti  Prograa  muat  bo  piT<v1ded  with  the  matricce  A,  B, 
and  C.  If  the  customer  does  not  care  to  pre-ccorpute  these,  the  Entrance  t,o  KLceati 
Routine,  say  be  xused  with  inputs  G,  H,  Q,  and  R  *  .  Ihls  progi'am  vl  i.l  compuu' 

A  «  H’QE,  B  -  OR'^G’,  and  R”^0’. 


5.  Cheeks. 

A)  The  Program  was  run.  with 


0  0^ 


where  0^  a  lx,  dimensional  zerj  column  vector  and  =  6  x  6  identity  rjitr'.x. 


Q  =  H  »•  T  7  ?.ero  matrix 


R  -  ro. 60^-51 

P(0)  »  0.6075  I, 

T  -  -  0.2. 
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!IhlB  vms  Iterated  for  ten  steps  said  the  reeui.t  compared  with  a  barid-<;otBput?td 
result  esepressed  exactly  in  four-place  decimals.  The  Riccati  printout  appears  in 
?ig.  1.  The  hand  cosqprrted  result  la! 


0.2025 

0.4050 

0.4050 

0.2700 

0.1550 

0.0540 

0.0180 

0.1>050 

1.4175 

2.0250 

1.7550 

1.0800 

0.5150 

0.1980 

O.J»050 

2.0250 

5.8475 

4.1850 

5.1050 

1.7280 

0.7650 

0.2700 

1.7550 

4.1850 

5.8275 

5.4450 

5.7170 

1.9680 

0.1550 

0.0800 

5. 1050 

5.4450 

6.6575 

5.8410 

5.8750 

0.05^^ 

0.5150 

1.7280 

1 

2.7170 

‘).amo 

6.3519 

=s,9j.72, 

0.0180 

0.1980 

O.7050 

1.9680 

5.8750 

5.9178 

6. 8' >25 

2)  The  Program  was  run  with 


*1  ’ 


[i 


G  =  R^=-7XY  zero  matrix 

Q  2.0250 
p(o)  ..  2.0250 
”  -0 « 2 

TiilB  was  It.n  fated  for  ten  steps  and  the  re  suit  compared  vd  t.h  a  hajid-cc-mputed 
rouuil  which  was  expressed  exactly  in  four  place  decimals.  Ihe  Kiccatl  prlt'.to'.d, 


eppearu  in  Fig.  2.  Hie  hand -computed  reau.Lt  ial 
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0.2790  -0.C*90C=  ■  0.0180 


-1.9758 

^.£869 

-1.9710 

1.2670 

-O.olkBo 

0. 21^50 

•^-05^0 

1.2990 

-1.9710 

2.2725 

1.9550 

1.2150 

-0.55»00 

o. 1550 

-0.6720 

1.2870 

-1.9550 

2.1825 

-1.7550 

0.9^50 

-0. 2700 

0.2790 

•0  61*80 

1.2150 

-1.7^ 

1.8225 

-1.2150 

Q.)t050 

-0.0900 

O.2U30 

-0.5HU. 

0.9‘^50 

-1.2150 

1.0125 

-o,i405>:! 

0.0180 

-(D.055»0 

0.1500 

-0.2700 

0-itf)50 

-0.ii050 

0.2025 
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1,  introductloPt  Tlwira  1b  no  doubt  that  Wiener’s  theory  of  statist  leal 
predletlon  and  filtering  Is  one  of  the  great  oontributloos  to  eagineerine 
science.  Yet  the  theory  has  found  few  practical  applications  so  far.  This  Is 
proibably  due  to  the  difficulty  of  neasujrlng  the  atatiatlcal  characterlBtlcs  of 
raadcB  peroeeBses,  vhlch  is  the  starting  point  at  the  theory. 

But  even  if  one  is  vllllsg  to  accept  physically  aotivated  aesuaptlonu  in 
place  of  e3qperl.vissbal  statistical  data^  there  rensains  a  Ba3or  j^poblem:  cc^ap^^- 
tatlon  of  the  aptUaal  filter.  Current  textbooks  [1-5]  contain  screral  wsthods 

f 

fotr  doing  this.  l%tese  nsthods  yield  analytical  answers  only  for  a  few  trivial 
aeadeiilc  exMSples^  and  they  are  rather  poorly  suited  for  nuaerlcal  coaputations* 
Most  of  these  procedures  texnlnate  with  the  iKpulse  response  of  the  optimal 
filter*  This  la  net  a  cosplete  sclutlcs;  of  the  prcblcv^ however,  since  there  is 
In  genera?  no  eliB(plje  method  for  synthesizing  a  filter  with  a  preocribed  lastpuise 
response.  Another  shortcofldng  of  the  conventional  approach  Is  that  the  treat»&nl 
at  time-variable  problems  Is  very  awkward. 

This  paper  Is  uoncesmsd  with  OTeTcoadng  dlffictiltles  of  the  second  type 
mentioned  above.  The  rsquirad  statistical  data  are  assumed  to  be  given  as  part 
of  the  problem  stateiBent.  Moreover,  these  data  are  given  in  such  a  form  that 
caaoputatlrm  of  the  optimal  filter  is  highly  slupllfled,  with  a  single  equation 
conrirlng  all  cases. 

The  Wiener  problem  is  reduced  to  the  classical  haalltonlan  formalism  of 
the  calculus  of  yariatlona;  many  Icmg-standlng  difficulties  of  the  theory  are 
resolved  or  greatly  clarified.  The  solvation  consists  in  the  opeclfication  of 
the  differential  equation  of  the  optimal  filter. 

The  reader  should  be  warned  right  away  that  the  Wiener  prob.lae  is 
raally  one  in  i»tatlstlC8.  It  belongs  to  the  realm  of  pure  probability  thasorys  It 
la  sLillar  J.n  ^c•ec  rays  to  ahe  law  of  Isua*  numbers,  central  limit 
etc.  Wiener's  aiproach,  as  ours,  requires  that  the  probabilistic  structures  of 
the  random  processes  be  known  exactly.  Therefore  confidence  llaits,  fttetiaticol 
decision  rules,  ate.  do  not  enter  the  picture. 

Wiener  aaauams  etationerlty  and  describes  the  random  porocess  by  Its  power 
spectral  density  or  covariance  function.  In  thia  paper,  we  asstme  Slightly  more: 


>4P^RiVi 


the  vandpm  process  is  to  he  TsarStnvlMis  ia  other  vojeua,  it  can  he  then^^t  . 
oi'  as  bejLwg  generated  hy  a  linear  dynaElcal  systoai  (of  finitely  many  degrees 
of  freedom.)  axoitsd  by  dhlte  noise,  Snie  is  very  nearly  the  only  case  where 
explicit  solutions  of  the  Wiener  problem  haTo  been  fovmd  Jji  the  pest.  Ver^ 
roughly,  Wltaner^B  point  of  riov  is  to  adalt  the  posBlhllity  of  clLenu!aeraTij:Ly 
infinite  degrees  of  freedca  this  I0  iaportant  in  some  cases.  But,  in 
engtne«jrlng  prohlemn,  our  aseuaptlon  is  frequently  rore  natiaral. 

Tr»  ttry  ease,  the  difference  betueen  tfcs;  cli^a^l.eal  point  of  viev  and 
ours  becooBS  is^rtent  only  If  one  'wants  "bo  form  an  eBtlmte  of  the  power 
8i)ectral  density  of  a  random  process  on  -the  basis  of  actual  neasuroments. 

Wb  shall  not  be  concerxsed  with  this  qticstlon  here  and  wlJJL  study  the  Viener 
problem  solely  from  the  standpoint  of  probability  theory. 

Bic  exposition  glren  here  suaniarlses  the  contrlbutionc  of  two  earlier 
papers  rV- 5j»  alxiiough  may  details  irill  be  different.  Biere  are  also  a 
nvmber  of  theoreas  and  exBS?pleo  which  appear  now  for  the  first  tinse. 

*  The  intent  of  the  paper  is  primarily  expository,  and  'we  shall  not  hcoi- 
tate  to  ocsdLt  certain  mathematical  technicalities  connected  with  the  rigorous 
deflftltlon  of  continuous  random  procesoea.  Brorythlng  cine  will  be  stated 
In  precise  mathematlcel.  'terns)  -the  reader  wishing  to  fill  In  the  missing 
de'balls  ahoxild  ha're  no  dlfricvLity  in  muXlng  contact  with  the  pure  laatheEaticai 
ll'teratura. 
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2<‘  of  Ooitt:&gat5.  ¥e  begija  with  «  ourrey  of  the  naia  toples  of 

tbs  ]pa2Mr,  lettriog  tbs  definitions  Intentioislly  soaewlMit  yttgiae  for  the 
itonent*  W»  give  here  also  an  aooount  of  the  SAneral  phlloso]^  of  approauh 
tiOcen  in  this  paper.  Oe  reader  night  read  i^ironi^  this  section  lli^ly  at 
first;  rotnmlng  to  it  after  a  detailed  st\»dy  of  the  suooeeding  natexlal. 
Starred  seotlons  ac4  exanples  nay  be  onltted  without  interrupting  the  logical 
sequsnoe  of  exposition. 

A  random  process  is  a  fnadly  of  functions  x(tj  ®)  depending  on  two 
arguoMnts:  (l)  the  tine  t,  ^ioh  is  a  real  nvonber;  and  (ll)  a  random  OTent; 
which  is  denoted  abatraetly  by  the  sytobol  a>.  If  o)  »  is  a  fixed  rsndcaa 
event;  then  x(T;  is  some  function  pf  time,  usually  called  the  sample - 

function.  If  i®  fixed  instant  of  time;  then  oo)  Is  a  randesa 

variable,  which  is  frequently  'written  simply  as  x(t^).  Instead  of  letting 
be  a  real  number,  we  can  take  t  to  be  am  integer;  in  this  case  we  call  the 
family  of  functions  x(t,  a>)  a  random  sequence.  A  random  process  or  sequence 
can  also  be  regarded  as  a  l-paromater  family  of  random  variable b. 

A  random  process  or  sequence  is  described  na thematically  by  cpecifying  (l) 
‘a  collection  (''ensemble'')  of  8anq>le  functions  {x(*,  cd))  and  (il)  the  probabi¬ 
lities  of  the  random  events  o.  In  general,  there  are  nondenumemblj'  mani' 
random  events  and  therefore  the  probability  of  a  single  '»V'  ‘  be  sot  pqual 

to  0.  One  gets  around  this  dliflculty  by  defining  the  probability  of  sets  of 
events.  A  rigorous  definition  of  rmndom  process  is  a  complicated  problem;  see 
Doob  [6, Chapter  1]  and  L^ve  [7,  Chapter  9]. 

Svqppose  that  we  have  observed  values  of  x(t,  «>)  corresponding  to  some 

interval  of  time,  say  t^  S  7  s  t,  lAero  t^  denotes  tbe  starting  point  of 

observations  and  t  refers  to  the  present  instant  of  time.  Let  be  the 

set  of  all  BaniDle  functions  which  agree  vlth  the  observations  made  during  the 

interval  t  t  s  t..  let  0.  be  the  set  of  all  a>'c  such  that  r-anpling 
o  t  ^  u 

functions  corresponding  to  them  belong  to  the  set  F,.,.  By  dividing  the  pi'O- 

c 

bttblllty  of  any  subset  of  itself,  we  obtain  the  conditional  probability 

of  the  occurrence:  of  sample  functions  for  values  t  t.  We  can  nerw  state 
the; 

PREDICTION  PR9BISM.  Given  the  actually  obeervad  values  of  a  nmdem 
proceaa  o%:er  some  Interval  of  time,  find  the  conditional  probabilities  of 
all  future  val\ies  of  the  random  process. 
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Biu0  tile  predlctim  probleaa  ccroelets  In  celeulatlng  conditional  prol)a- 
■bilitlee  —  often  of  a  irery  otnpllw  type- 

tte  filtering  problm  is  vexy  similar:  instead  of  observing  x(t,  w). 
se  observe  a  randM  proeest  z(t,  <»*)  related  to  3c(t.  a>),  the  pro¬ 

babilities  of  sets  of  erents  «  and  a'  are  dependent  cn  one  another.  For 
easBipIe^  x  flay  be  a  signal  and  e  the  signal  plus  noise.  Stated  fonoallyt 

TXLOBRSn  HiOEEEK.  QlTSn  the  actially  observed  yaluee  of  a  random 
nrocesa  over  soae  interval  of  tjaa,  find  the  coodltloaal  probabilities  of  all 
raluea  of  another*  related,  raodon  process. 

Qcoe  the  conditional  probabtUtlea  are  knovn,  one  can  of  course  ansver 
in  principle  any  problem  concexning  the  probable  futuie  evolution  of  the  pro- 
cees:  the  conditional  probabilities  incorporate  all  the  inforaation  inherent 
In  the  observed  values  of  the  random  process.  Bie  adjective  "optimal"  as  used 
in  prsdictlcc  and  filtering  theory  refers  to  the  fact  that  all  Information 
contained  in  the  observed  data  i.^  taken  into  account. 

At  present,  there  Is  but  one  class  of  problems  In  vhlch  the  prediction 
or  filtering  problems  can  be  effectively  solved:  both  x  and  z  must  be 

gauss-oarkov  processes.  In  this  case  the  solution  is  quite  DlK5)le  in  principle. 

« 

Bie  conditional  jprobabilit)  distribution  of  a  gausslan  process  is  completely 
described  by  its  mean  values  and  covariances.  If  in  addition  the  process  la 
also  narkorian,  than  It  suffices  to  knov  the  means  and  covariances  f.  ’  ric 
instant  of  time. 

Ihe  solution  is  as  follows.  We  must  conqnitc  the  conditional  nxinns  and 
the  conditional  covariances.  *1110  conditional  means  will  depend  on  the  observed 
values  of  the  z  process.  They  axe  computed  by  putting  the  observed  value b 
throuc^  the  eo’^all.od  optimal  filter.  Bie  conditional  covtirlancee  are  inde¬ 
pendent  of  the  observed  values*  Therefore  they  can  be  compiuted  neps^ratciy, 
even  before  any  observations  have  been  mwle.  Khowlfidgp  of  the  conditional 
variancea  at  time  t  m  t  is  necessary  to  compute  the  conditional  means  at 
time  T  t,  TSie  equation  for  the  condlticnal  mean  is  linear,  the  equaticn 
for  the  covariances  is  aonllneax. 

We  digress  momaiitarlly  to  enq^haislze  seas  consequences  of  the  mrkovlcin 
and  gausslan  a88UBq>tlons. 

Any  randjorn  pjrooesB  may  be  regarded  as  mrkovian  with  a  suitable  defini¬ 
tion  of  the  state  of  the  process.  For  Instance,  the  state  nsoy  be  taken  a» 
the  obsenred  past  hlotoiy  of  the  prxx^ess.  The  Inq^ortant  thing  is  to  find  the 
"smallest"  state  space  for  which  the  markovian  property  holds.  We  phall 
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assume  hBx'e  that  the  state  of  the  process  can  he  da8crll«(i  hy  a  Tector  -with 
finitely  jaany  coajxiinentsi  In  other  words,  the  state  spate  Is  finite  ■■diraBnBliaial . 
!Qil8  assumption  la  highly  desirable  becauae  It  leads  to  differential  oq^iatlona 
of  finite  order  vhich  oan  be  treated  by  standard  mcthcjds.  Ve  cotsld,  for  ths 
sake  of  greater  generality,  oj^rate  in  a  "larEer”  (i.e,,  Inflnlto-dlnensiooal) 
state  space,  but  the  nathanatlcal  subtleties  -mich  arise  do  not  coeta  to  uaTe 
physical  signlflesnoe.  And,  after  all,  physical  systems  can  be  (and  often  must 
be)  {^pproxlsiated  by  differential  equations  of  finite  order, 

Ohe  rsasens  for  ths  gausslan  assuapticn  call  for  longer  ejqgaanat-lon^ 

EUnce  the  strict  prediction  or  filtering  problems  cannot  be  aolTcd  la  gonaral, 
it  Is  satunl  to  take  a  look  at  veaker  versions  of  the  same  problem,  For  in- 
atanee,  let  us  consider  the:  « 

maiAR  FIIffiiRlTKi  ?«OBUOI.  Find  an  eatiaate  x(t^)  of  x(t  vhloh  la 
(1)  a  linear  fhactioo,  of  the  obaerred  valusa  of  e(t)  and  (ii)  iBlnlwiseg  the 
TBBan«aquare  B{|x(t^)  -  x(t^)|^. 

It  turns  out  that  the  solution  of  this  problem  la  identical  with  the 
aolutlon  of  the  (strict)  gausaian  filtering  problesi;  the  optimal  sauaaion 
fU.ter  Is  a  linear  flltar  which  mlnladxeB  the  mean-sqtaira  error, 

She  proof  of  this  dual  intorprotatiesa  of  the  optimal  filter  la  the 
following.  If  we  seek  the  boat  linear  estimate,  then  only  the  first  cad  eaecnd 
order  ooqKmts  (l.e,,  the  means  and  comrlencea)  of  the  z  -DroceBo  need  to  be 
known,  Olren  any  raiidom  process  with  prescribed  meana  and  corarlancea,  cne 
can  find  a  unique  gauasion  process  wltlx  the  same  »B«mB  and  comrlaneeB.  (!iailo 
lo  trlylal  since  a  gausaian  random  process  is  uniquely  deteitdaed.  by  its  mesna 
and  coYarlences, )  Hence  the  solution  of  the  liKaa-r  filtering  problesi  in  the 
gatisaiflR  case  Ktust  be  the  same  as  In  tbe  general  case.  But  In  the  gausalnn 
case  the  solution  of  the  linear  filtering  problems  Is  alatultanecunly  alao  the 
solution  of  tbe  strict  filtering  problem*.  This  la  because  In  the  jausBisn 
case  thr:  <baaj:-«>^vwirG  error  is  mlnlssircd  by  ths  coniltiona?.  rhich  («;«  we 
bare  noted  before)  Is  cooqmted  by  ueaxia  of  a  linear  operatiaa  on  the  z  procces. 


Bala  and  slallar  obaerfatlans  hai-w  led  Doob  [6,  pp,  71-^8]  to  introduce  tho 
notlca  of  "strict  boium"  and  "wide  sexiae"  prospertics.  oonnepte  are  sol¬ 

vated  aa  follows!  Oqipoae  a  xandoei  process  has  a  tsartv^ln  prtqperty  3f  which  can 
be  expressed  In  terms  of  means  ami  oorymxljenceB,  Suppose  also  that  the  unique 
gaueelan  process  with  the  same  means  and  oorarlaucfts  has  a  aorrs^pending  but 
•trenoer  property  .  Bban  P*  Is  a  ytrlot-sense  and  P  Is  a  viiiyg- 

sense  property.  In  particular,  the  filter  i^ch'  is' opijlBBl  la  the  strict  sense 
In  the  gauaelsn  case  has  the  wlde-sense  property  that  it  is  the  optlwal  lineig. 
filter,  without  sny  asmaqiktiosi  of  gausslanness. 


IX) 


It  is  a  aattet’  of  tsis^e  vblch  of  the  two  q\iestic«iB  we  pose;  the  answer 
ic  always  tb©  ^aaie.  If  w»  demand  a  strict  answer,  we  must  also  accept  the 
higjily  restrictive  gauBSlan  aBsunrotlon*  If  wo  look  onlv-  for  the  best  linear 
filter,  then  knowledge  of  the  first  and  second  moments  of  the  naidom  ptocess 
suffices  and.  nothing  more  has  to  he  assumsd  about  the  natvire  of  the  probahi- 
llty  distrihutlcn.  To  put  it  differently,  if  we  know  only  the  first  and 
second  aaoenta  of  the  random  process,  then  we  have  only  the  first  (linear) 
apprcKlsatlon  to  the  dynamical  model  for  the  prccena.  (Kiowlcd.ge  of  only  the 
first  mouiontB  would  give  the  zero-th  order  approximation,  i.e.,  a  mcjdol  which 
is  neither  dynamic  nor  stochaatlc.)  Virtually  nothirg  is  known  at  present 
even  about  the  second-order  approotlmation. 

We  now  turn  to  a  description  of  the  ta^thematical  results  sect ion -by-section. 

The  concept  of  a  gavLSS-markov  sequenco  is  introduced  In  Sect.  5.  We  u-jc 
as  the  haslc  definition  a  linear  dynamical  system  excited  by  a  gaussian  white- 
noise  sequence  This  is  physically  appealing  and  avoids  the  unnecessary  restric¬ 
tion  to  statlonarlty. 

Tbe  gaviBS-markov  process  is  defined  nlmllRrly  in  Sect.  6.  This  Invo.Lven 
an  unpleasant  technical  difficulty  because  it  is  not  possible  to  give  a  rigor- 

II 

oua  definition  of  the  gausaien  white-noise  process  by  elementarv  means.  We 
give  here  only  an  intuitive  definition  in  terms  of  an  (improper)  limit  nrexioss. 

A  rigorous  definition  via  generedlzed  1uui;o^.xib  Is  nowadays  fairly  straight¬ 
forward,  as  may  he  seen  from  the  literature  cited. 

Sect.  7  is  concerned  with  showing  that  the  rcprosentatl-ons  of  the  gr.ueo- 
markov  sequence  and  gauss Hoarkov  process  given  in  Sect.  5  fnid  6  con  deduced 
by  postulating  merely  the  guassian  and.  markoviaj)  properties,  lienee  rei>reeent- 
Ing  such  processes  as  the  mofilon  of  a  linear  dynamical  system  acted  on  by 
idxlte  noise  is  not  a  loss  of  generality. 

Sect.  8  introduces  the  main  topics  of  the  pni)er  by  a  t  Vgoroua  but  e3.enien- 
tar.  diacuasion  of  a  standard  prediction  piublea.  This  pi’ovldeo  an  Intoroctir.K 
comparison  of  old  and  new  methods. 


A  precise  statement  of  the  filtering  problem  with  which  the  impci  lit 
cohceroed  epx>earB  In  Sect.  9;  together  with  a  discussion  of  sobs  trswlitlona? 
problems, ail  of  which  reduce  to  our  filtering  problem  by  a  suitable  choice  of 
notation  or  by  minor  auppiementsjry  assumptlone.  Equationa  (l. )  and  (l  }  uenutc 
the  model  of  the  imndcKa  sequence  and  process. 

Since  the  prei  at  problem  formulation  is  unconventional,  its  relationH 
with  more  standard  formuiations  >rre  explored.  In  Sect.  10.  Unfortujii\tely,  tb'? 


ataadajrd  approAoh  to  filtering  rather  tends  to  obaeor®  the 

theoretlo&l  leatjes  Involved*  l!h«  min  point  ia  that  neein-8q[dftre  iptisal 
linear  filtering  1#  optimal  alno  vlth  respect  to  mny  criteria  other  than 
mean  8<)>iare. 

£teet«  11  is  the  solution  of  the  optimal  filtering  problem  vben  the  tlse 
is  discrete.  Vbtls  Is  an  improrveiaent  over  the  presentation  In  achieved 
1^'  the  use  of  the  so-called  psaudo-inverBe  of  a  matrix.  Ve  sasphaBizo  strongly 
that  a  finite  nxxaber  of  paxmaeters  (the  candltlcnal  mean  and  the  conditional 
Qovarlenoe  matrix)  can  be  regarded  as  the  "state”  of  the  filtering  problem; 
the  resulting  simplicity  of  the  solution  ie  due  solely  to  this  fact.  The 
principal  equations  of  the  tharcry  In  the  discrete  case  aret 
optimal  filter  and  the  variance *equar, Ion.  In  Sect.  12,  give  two 

examples  which  illustrate  in  detail  the  mathematical  nnd  also  the  physical 
significance  of  these  equations. 

In  Sect.  i5»  ve  obtain  the  continuous  analogs  (H^)  tuid  of  the 

optimal  filter  and  of  the  variance  equation.  Because  of  the  difficulty  in 
giving  a  rigorous  definition  of  random  differential  equationB  when  the  excita¬ 
tion  is  a  white-noise  process  we  do  not  give  a  rigorous  derivT’tion  of  these 
results  but  apply  the  inqiroper  limit  argument  —  already  used  to  define  Lin; 
white-noise  proceBS  in  Sect,  6  —  to  deduce  (ll^)  cuid  (iXX^)  from  sJ'd 

(m^).  Again,  a  rigorous  derivation  requireB  the  use  of  generalized  fuiictlone 
or  other  advanced  tools,  The  some  equations  (H^,  were  obtained 

before  in  a  different  way  [5].  Since  the  variance  equation  (lll^)  is  nonlinear 
even  the  proof  of  the  existence  of  its  solutions  Jb  nontrivial  —  but  easy. 

Once  this  is  established,  w«  see  that  the  varloncc  eqmitior  can  be  related  to 
a  hamlltonlan  system  (IV^)  of  2n  first-order  linear  differential  equatlono 
familiar  from  the  calculus  of  vorlatlona  or  fror;  tlieorctical  phybica.  'Rie 
latter  COT)  be  solved  more-or-lesc  oxiO-icltly:  this  ig  imi>ort-ant  beca’ane  one 
theireby  avoids  having  to  solve  the  varlanco  e  ioationo  by  nunierVcal  quadra¬ 
ture,  which  would  t)e  quite  \tjpleas«,nt  becanee  of  the  mi-ii'  variables  involved , 
This  surprising  and  y>at  natiuns-l  coruaectlon  between  the  Wienei;  fll  lerbig  ar.c 
the  calculus  of  variations  was  first  pointed  out  by  ifclroar.  ojid  Bucy  In  [^]. 

This  opens  up  rjany  promisjjng  possibilities  of  reaearch.' 

Sect,  lb  applies  (l^-TV^)  to  a  wide  variety  of  problem)!.  explicit 

ctepe  Hi-e  elementary  but  at  tljnes,  pai-tlcularly  in  Kxnmplc  (  Ul-20),  very 
intricate.  Some  of  these  problems  ere  among  the  moot  coraploc  ever  nolved  In 
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Vl<MB8r  fUterlng  theoxy.  MdltioctAl  axunqples  axe  gl'*^  In  [^]. 

fflb®  tery  great  dlffleultiee  encountered  In  the  direct  ana?  explicit  eoiu- 
tion  of  filtexljQg  prohliiM  literally  enforoe  a  change  in  point  of  Tlev.  Be- 
•Igned  to  the  fact  that  explicit  enavexe  can  only  he  o^4alaed  hy  ntseerical 
00fl|iutati0Of  one  uents  to  hay*  at  least  e  good  qualitative  understanding  of 
the  filtering  prohlem,  particularly  aa  far  as  the  dynanlcal  behavior  of  the 
▼arianse  etnaation  ie  oonoemed.  *'^ls  is  indeed  the  chief  task  of  flltei'lng 
theory  —  in  the  opinion  of  the  writer. 

Ito  Sect.  15  the  anich  slapler  prchlcr.  cf  uuulaoeu.  pam- 

eeter  eetlaation  is  considered  from  this  ]point  of  view.  We  obtain  an  Isqpor- 
tant  erltarlon  —  oooqplete  obaerrabllity  —  for  the  existence  of  ouch  cm  esti¬ 
mator. 


This  is  then  used  in  Sect.  16  to  prove  the  Teost  iuiportisnt  theoreni  of 

the  paper,  concerning  the  existence  and  uniqueness  of  limiting  solutlono  of 

the  variance  equation.  Finally,  we  eidilblt  a  canonlca.1  forro  for  tbe  Hn.talltonian 

equatiens  (IV  )  which  can  be  regarded  as  a  genojmlltatlon  of  Wlenor'o  well- 
c 

known  mel^od  of  ei>ectral  factorizatl»>n.-  follows  in  particular  that  If 
the  steady-state  solution  of  the  Wlex^r  problem  Is  known,  a  coBcg')lete  solution 
of  the  same  problem  with  the  observation  int.-'rval  being  f1.nl to  can  be  ccsistraetcd 
using  only  elementary  algohraic  steps. 

tOtiroughout  Sects.  13-16  we  avoid  appealing  to  the  duality  relatlo'.iB  which 
exist  between  the  optimal  filtering  and  the  optimal  control  problciu  [i^-5]; 
the  inroofs  are  given  by  direct  arguments  whenever  poBslble. 

itopendlx  A  presents  some  relevant  facts  concomieg  the  pseude-invorue 
and  generalized  inverse  of  a  aBitrix.  Appendix  B  1b  a  convenient  sunsnBry  of 
the  properties  of  gauaslan  raiidom  vocters.  A  noteworthy  feature  is  a  new 
forauLs.  for  cc;5dltior.al  sxpeotaticn  which  ic  valid  even  If  -!•!-'  ittn-lcv: 

vectors  Involve  have  a  sinTulc.'*  cora.rlir.ee  seetrix. 
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3.  Hietorlcftl  raBBffkg  and  anlmovied^gMneatB.  A  cfearacterlatie  re*t-ure 
of  this  paper  is  consistent  adherence  to  the  '*t is* -domain"  point  of  view.  In 
the  IpHO’s  and  early  1950  *s  aost  of  the  englnesring  litemture  in  prediction 
and  filtering  theory  vac  written  from  thr  ''frequency-domain"  point  of  vlev. 

This  was  in  haraocy  vlth  the  fashions  of  system  analysis  then  pro-vailing,  and 
can  he  explained  by  the  fact  that  most  stodiastlc  problemB  in  engineering  at 
•that  time  arose  in  -the  field  of  cOBBamicetiooiB  where  the  ’‘frequancy-dosmin" 
description  of  systems  is  qui-be  natural.  However,  the  rrerpijency-dougiiri  me-bhod 

it  nov  stands  is  not  veil  sui-tcd  for  ■the  s-tudy  of  ncailineax  sys-bems  or  even  linear 
sys-bems  vi.th  tlme-^vararlng  paroae-bers,  Progress  In  the  lat-ter  fields  lias  ra- 
awtenod  interest  in  "tine •domain*'  me-thods. 

One  of  -bhs  first  effective  solutlcas'  of  a  time-variable  filtering  problem 
vas  gtvvm  about  195^  or  1957  by  ahinbrot  to].  Al-thcugb  his  resulta  can  now  be 
obtained  more  easily  by  other  methods  (see  Sect.  ),  Shinbrot’s  work  contri- 
bu-bed  aubs-tantlally  to  a  bet-ber  \indera-banding  of  -the  tlme-varijable  fil-fcerlng 
problem. 

Ccjncurrently  or  perhaps  slightly  earlier  Pugachov  began  a  sys-tomatic 
ptxidy  of  tine -domain  me-bhods,  culminating  in  his  oxosllsnt  textbook  [9],  now 
in  second  edition,  which  :ls  acJjl  iitiJ/?  '-~Twn  outside  the  Soviet  Union. 

More  recently,  in  a  serine  of  .nqwirtant  papers.  Parses  [10-12]  has  laid 
-the  foundations  of  a  general  -theory  of  s-tatistlcal  estiaatlcn  by  cdSidina-be  -free 
me-thodsj -bu&t  is,  ImSapandantly  of  the  particular  hiLlbsrt-space  reproBfixi-taticn 
of  the  random  xnrocess. 

Very  crudely  speaking,  our  ajiproaoh  Is  the  most  effect  i-vo  but  also  -the 
least  general  cf  -ths  t-wo,  Paraen's  -wox’k  occupies  -the  other  extremej  Puffcchev 
is  dLn  the  adddla.  Barzcin's  and  Pugacbev^s  s-tarting  aasissTtissiiJ  llo  closer  to 
experlmen-tal  data,  but  -thtj  calculations  wiiich  -they  must  pexforat  to  get  explicit 
aasunra  are  more  Imrolred.  IJltlma-toly,  sosss  synthesis  of  -the  three  m?thcic  io 
likely  -to  eroive. 

The  other  oh&racterlstic  featiire  of  -this  paper  is  the  insistence  tl 
prediction  end  fil.terlng  is  primarily  the  detoxmiQ&tioGi  of  coadltlcsfLl  distri- 
butlcns  and  oisly  seoondarlly  the  orwputation  of  certain  fuaotlomls  of  ike 
eondlticanl  dictrlbirtlcns.  ISsls  point  of  vio-ir  is  now:  It  la*  been  forcsful.ty 
brought  forth  In  -the  work  of  I^xrstax^rs  tljl*  Surely,  this  is  ths  clsarsst 
and  most  ootrraniaint  way  of  studying  prediction  and  filtering  of  gauiBslan 
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pr-ixsesei  e\j2»3y,  this  vill  ba  the  startljag  polirt  for  rut'ore  studieB  in  non¬ 
linear  prediction  and  filtering  theory. 

Bhls  cxpoaltlon  vaa  prepared  ylth  the  partial  axqiport  of  the  U.  S.  Air 
Force  under  Contracts  AP  lt9(638)-582  and  AT  53(616) -6952.  The  vriter  is  in- 
dolrtod  to  his  coLLaaguea^  particularly  R.  S.  Bucy,  for  nussrova  Btimuliitliig 
comraraationa. 
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tha  notations  and  tandnoloiQr  of  1^]»  6aall  joIdLface  lottaris 
^  ••■>#£  denote  Tectors  vlth  coordlna+as  ...,  z^. 

Roosn  and  OreeSt  capitals  G,  ?,  ..-j  T  danots  antrlcas  ^osa  ols- 

Bents  az«  srlttsn  as  c. 7.  ..  Tb»  vsiix  Tsattrijr.  Is  I.  SaeOJ.  Greek 

ij  ij  - 

letters  usualljr  denote  oonstants.  Tbe  tl«s  is  denoted  Isy  t,  t  ,  .  or 

Tj  these  Mgr  he  arbitrary  real  nusfbers  (e<»tlnwms~tl*B»)  or  arbitrary 
Integers  (dlacrete-tlBB ) «  Tha  letters  1>  q  are  reserved  for 

Integers. 

The  tranapcae  of  a  taatrlx  ia  denoted  hy  the  prJ«B.  Inner  prod>ict 

f 

of  X  and  denoted  by  x*^  and  the  tensor  product  by  idilch  la 

Just  a  natrtx  vlth,  elenanta  noT«  jjxij  ir.  If  A  ia  a 

sysmstrlc,  ncnnecgatlve  definite  natrlx  ve  use  the  abbrevSntlon  ||xt|^  for 
■che  quadratic  for«  x'^.  IJfunerlcal  quantities  wlLl  be  olvaye  real,  ne-? 
coiiq)lex. 


IRunerlcal  quantities  will  be  olvaye  real,  nerer 


The  symbol  E(  }  denotes  the  expectation  operator  (or  enaeisble  average). 
We  •shall  retain  the  curly  brackets  for  greater  clarity  even  if  several,  symbols 
E  ate  used  In  the  oaine  fonrtuij*.  '  ’“+ls?efl  w  ^rrite  covarionce  isatricco  as 

corv[xl  «  E((x  “  E[x))(x  -  E(x})'} 


cov[5^  JC]  =  K((x  -  -  KCj;:))*) 


A  contlnvous-tliaB  linear 


syutem  in  tlilB  rooaij  the 


Byntem  of  eqiaatious 


d^dt  y{t)x  +  tl(t)u(t). 


y(t)  =  H(t)x(t)^ 


(4.2) 


and  a  dlserotc-tlme  linear  dynamical.  oy»tata  will  be  the  systera  of  differeaco 
equations 
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s(t  +  1)  »  |{t  +  1,  t)x(t)  r(t  +  1,  t)u(t); 


(i^.3) 


iCv)  »  H(t)x(t). 


(kA) 


3ii  both  oasea^i  ue  call  the  n-^fector  x  the  atate  of  iaua  syetes, 

B-Tsctor  tt(t)  is  the  Input  or  control  i\ffjctlcei,- ana  the  p-v»ctor  j;  In 
the  output.  Iftieca  the  input  is  an  unocntroUablti,  say  rendoa,  quantity^ 
m  replace  u(t)  by  w(t).  GL»  K  reap,  tj  5  are  n  x  n,  n  x  a, 
p  X  n  ffiatrlces.  If  all  thOM  satrloea  Skre  constant,  then  the  ie 

said  to  be  constant  (or  utatlocaryH  If  u(t)  -  Q,  then  the  eystea  la 
free. 

. .  f 

5he  general  aolixiion  of  (^.l)  is  Imoun  to  be  [I?  ja 


t 

x{t)  «  ♦(t,  t^)x  +  /  ♦(t,  t)0(T)u(T)dT 


vltb  arbitrary  3^  t,  t^.  Cils  formula  la  valid  if,  for  Instance,  u(t)  io 
a  contlnuoua  function,  in  which  case  the  function  x(fc)  defined  by  (U,5) 
has  the  foUovlng  propertleB: 


(l)  it  satisflea  the  Initial  condition: 

(il)  it  is  differentiable  and  eatlsfles  everyAere  the  dlfferontial 
equation  (4.1); 

(ill)  It  is  \jniqucly  detoralned  by  the  choice  of  ^ 

!!hc  matrix  occiirrlng  in  (4.5)  la  called  the  titmaltlcyi  ioatrix 

of  (4.1);  It  \B  charecterlred  by  the  propertlea  *. 


®(t^,  t^)  -  I  for  all  t^,  (4.6) 

(thla  follovB  from  (l|; 

♦{tg,  t^)*(t.j  ,  t^)  ♦(t^,  t^)  for  all  (4.7) 

^  (this  foUovTB  from  (4,5)  and  (iil)'': 

In  addition, j^satlBflee  its  om  differential  c-inntloc 
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-  F(t)i(t,  t^)  (^+.0) 

(this  foUcnrs  \xy  setting  u(t)  =  0  and  then  differentiating  (4.5))-  Frf>r.i 
(47)  it  is  claar  that  i  is  never  singular. 

It  can  be  shovn  tbax  properties  (4. 6-4.8)'  unigiteily  determine  the 
transition  matrix  of  the  differential  eouatton  (4.1), 

When  r  is  constant,  the  tiransttion  matrix  depends  only  on  the  differ¬ 
ence  t  -  t  and  can  be  explicitly  defined  as  the  exponential  of  the 
o 

matrix  F; 

00  . 

0(t,  t  )  -  exp  E  (Fi(t  -  t  )]  /l’  (^.9) 

When  F  is  not  constant,  there  is  no  simple  way  to  compute  $  explicitly. 

Turning  now  to  the  definition  of  a  diocrete-time  linear  dynamical 
system,  it  is  not  necessary  tc  assume  that  ^(t  +1,  t)  la  nonalngular. 

But  it  Is  virtually  no  restriction  at  all  to  vAd  thin  aaoimiptlon;  then  or.c 
can  define  by  induction  ^(t,  t^)  so  tliat  relations  (4.6-7)  satisfied 
for  all  Integers  t, 

It  la  easy  to  reduce  fh.l)  to  (4.5).  As  far  as  the  transition  matrix 
is  concerned,  ve  simply  coiislder  It  for  Integei-  values  of  time  only.  Wc 
must  aseume,  however,  that  u(t)  is  piecewise  conotaut,  that  ic 

’j(c)  «  u(k)  when,  k  4  t  <  k  +  1,  where  k  -r:  integer. 

Then  the  Integi-a?  in  (4. 5)  can  be  computed  explicitly  and  we  find 

t+1 

r(l  -i  1,  t)  /  ^.(t  1,  •i)r,(T)dr,.  (^).in) 

t 

The  converse  Is  not  time,  os  may  be  senn  at  once  by  crmnidcring 

x,(t  ►  1)  =  -  x^(t),  (4.11) 

since  the  number  —  1  does  not.  have  a  reai  3og.«irlthm.  In  othsx’  worda ,  It 
is  impossible  to  '’imbed"  (4.11)  In  a  ccRiitinuoua-tlmo  d^ammlcal  eystem  vlth 
a  reel  one -dimensional  state  space. 
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A  linear  dynasalcol  eyete®  is  said  -U.  be  atabLe  if 

^o)l!  *  a  for  all  t  s  t^. 

It  Is  asynaytc-tlcsLUy  stable  In  addition, 

Ite  ||i(t,  tjtl=0  for  an 
t  -♦  oo 


(4.12) 


(4.15) 


Finally,  the  system  Is  uniformly  ajygDtotlc.ally  s  cable  If 


t^)ll  <  006 


-^(t  -  t  ) 


for  all  t  k  vhore  Of,  ^  >0. 


/■!.  ^  \ 


Ibese  definitions  follow  by  spoclelizing  the  BK>re  general  deflnltloais  for 
arbitrary  (posoibly  nonlinear)  dynamical  oyoteans  [i4  ].  For  a  cor-.nt^r,t 
system,  (4.l4)  la  equlTal-Cnt  to;  all  (^Igepyalues  of  F  haye  ncf^tlve  real 

parts. 
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5»  Bae  g«u»«-<a*rkov  sequgnee.  ISiiH  1«  «  saquacce  of  random  vectors 
x(t).  x(t  +  1),  .*•  generated  by  the  recurs  Ion  raJatlor; 

x(t  t  1)  -  ^(t  +  1,  t)x(t)  +  rCt  +  1,  t)v(t),  (5.1) 

ehera  w(t  ),  v(t  +  l),  ...  Is  a  sequence  of  gatiBeian  random  vectors,  any 
tvo  of  vhieh  taken  at  different  ar»  indejvendent.  By  gausslanness,  the 

last  property  is  equivalent  to  the  vanishing  of  the  cross -vaxlence  matrix: 

cov[w(tj^),  wCtg)  I  ^  Q  if  /  ■^2* 

f 

^ou^  not  logically  necessary,  for  the  purpoees  of  this  paper  It  will  be 
assisoed  that  the  sequence  v  has  zero  mean: 

E(v(t))  -  0  for  alJ.  t.  (5.5) 

Cien,  by  gausslanness,  it  follows  also  that  the  sequejnee  w  1  r,  unlq’icjy 

determined  by  its  auto-covarlance  matrix; 

cov[w(t)]  =  a(t).  i^.k) 

It  should  be  noticed  that  this  definition  lo  not  complete  \citil  tne  ini¬ 
tial  state  ^C'^q)  of  the  dynamical,  system  ^5*1-)  lo  spec'' fled.  It  1q  natiural 
to  assume  that  i(tg)  is  a  random  vai’lahle.  In  fact,  a  ^ausolan  random 
variable,  with  zero  mean  and  arbitrary  variance,  liwlependent  of  v.  yince 
linear  combinations  of  gaiuisian  random  variables  arc  gaunnian,  it  fo]  Ijv^a 
that  x(t^),  l)^  •••  la  a  sequence  of  p^uasian  random  vai-lah.l-s  with 

zero  taean. 

By  repeated  application  of  (5.1),*  write; 

x(t, )  -  ♦(t-,  t,)x(t  )  +  E  ®(t,,  t)r(t,  t  -  i)wrt  -  ij. 

t  o  .,  X 

f?lnce  the  w(t)  occurring  at  different  tinei"  aiv  Irrlii^cndent ,  it  fol 

that,  for  t-  >  t  , 

'  1  o 


(5.6) 


Pr(Xj^(t^)  i  ii#  •••#  x(tp  -  1),  ...  ) 

■  S  1^,  *•*»  ^ 


In  other  vorde,  the  conditloual  parobability  difltrlbutlon  of  i(t^)  given  xCt^) 
•hd  preceding  observed  values  of  the  state  veriable  is  identical  vith  the 
probebillty  distribution  of  ac(tj^)  given  the  last  observation  x(t^).  Hela- 
tion  (5.6)  la  usually  called  the  (atrlct)  eiftrkov  property, 

Ve  have  now  Justified  the  xise  of  the  adject ivee  "^susa"  and  "iHirkov"  vith 
the  sequanee  generated  (5.1). 

analogy  with  the  ccnsaon  usage  In  randora  procesaea,  ■vq  my  c&H  v  e 
(aauselan)  iddte -nolee  eeQuenee.  Thus  a  gauss -ejarkov  randoa  sequence  ia  a 

f 

dlBorete-tlMB  linear  dynswi,eal  system  excited  by  gausaian  vhlte  noise. 

The  sequence  (j.l)  serves  as  an  ideoJLlaed  linear  model  for  random  pro¬ 
cesses  observed  in  nature.  In  general,  the  state  x(t)  of  (5.I)  Is  an  abstract 
entity,  not  axnenable  to  direct  physical  measurement  =  To  raako  the  model  more 
realistic,  we  add  the  assuaqptlon;  all  observable s  y(t)  are  ac  linear  func¬ 
tions  of  x(t).  Thus  we  adjoin  to  (5.I)  the  equation 

V  ' 

z(t)  *  H(t)x(t)  +  v(t)  «  y(t)  +  v(t)  (5-1') 

whore  ▼(t)  is  a  white -noise  sequence,  specified  by 

.cov[v(tj^),  vCtg))  -  2,  if  tj^  /  tg, 

®(2l(t))  “  2  *^bl  t, 

CCiv[v(t)  ]  ■  R(t), 


v(t)  to  x(t)  •  H(t)x(t)  is  intended  to  reflect  the  fact  thi't 
physical  mcasurcTKinto  of  obsorvables  can  never  be  iflode  wii-h  infinite  pr«clsior.. 
¥e  shall  reserve  a  detailed  motivation  and  critique  of  this  aasumptlon  Uiitll 
later. 


Evidently, 
(more  precisely, 
pend  on  x(t), 


s  given  by  (5.I')  is  a  random  sequence  >dilch  is  z-elated  to 
conflated  with)  tlrie  sequence  x.  Hot  only  does  5.(b)  de¬ 
but  there  may  be  a  correlation  bottfeen  w(t)  and  v(t): 


and 


eoY[]r(t),  ift)]  ■  C(t) 


(5^7) 

Vs  shall  adopt  (|J.  1>1')  as  the  staodaurd  fona  of  the  gauss -markor  sequence 
Any  guasslan  iiiiite -noise  seq:uenco  can  be  put  Into  this  fom,  as  ve  shall  peers 
in  Sect,  7. 

Bje  system  (5. 1-1’)  is  shovn  schematically  In  yig.  1.  ©lis  is  a  conven¬ 
tional  block  diagram,  except  for  the  fact  that  the  rectangular  blocks  denote 
matrices  (not  scalars})  the  signals  are  vectors.  To  differentiate  Fig.  1 
from  scalar  block  diagrams,  the  signal'  flow  is  depicted  by  fat  arrows. 
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6.  ah»  gfcvms-wtrkov  proceiis.  Intuitively,  this  concept  Ib  most  re»A\'‘-y 
undaretood  ac  tba  limit  ins  case  of  a  gaue«>markov  eaquence,  vhen  the  dlstacsce 
betVBon  suceeBSlva  values  of  time  tends  to  sero.  We  have  eLlrecidy  noted  In 
Sect.  tliat  —  if  the  foroUig  function  is  plecevise  constant  —  any  linear 
differential  equation  tsay  "be  converted  into  a  linear  difference  equation  in 
such  a  vay  that  at  Integer  values  of  time  the  solutions  of  the  differential 
equation  agree  with  the  solutions  of  the  difference  equaticn.  Bils  procedure 
vlll  now  he  used  in  the  reverse  order 

Let  us  replace  (5. 1-1^  formally  by  the  system 

dx/dt  -  r(t)x  +  0(t)v(t),  (6.1) 

»(t)  -  £(t)x(t)  +  T(t).  (6.1*) 

Bie  block  diagram  of  this  systom  is  shown  in  Fig.  2j  the  box  l/f,  syw- 
bollzeo  intogratlosa  with  respect  to  tiue, 

Ihe  terms  v  and  w  ;'n  (6.1-1')  should  be  llml^yng  cases  of  the  gauB- 
aian  white-noise  aeq^iences  denoted  by  the  same  symbols  in  Sect.  5.  The 
problem  *-  tn  wake  this  notion  precise. 

First  ve  define  the  random  processes  v  ard  w  In  such  a  way  that  at 
Integer  values  of  time  the  random  processes  x  and  z  generated  by  (6.1-1') 
agree  with  the  random  sequences  x  and  z  generated  by  (^.1-1'). 

To  accomplish  this,  the  saiqplo  functions  are  to  be  piecewise  constant 
«?ver  IntervalB  of  length  1.  We  set 

■  l(*^)  '“wi  ]?(^)  "  (6.2) 

whs  re 

k  «  integer  and  »  t  <  k  +  Ij 

the  ri^t*qiand  sides  of  (6,2)  are  to  be  the  gaussien  white -noise  sequeaces 
denoted  by  the  same  letters  in  Sect.  5.  See  Fig,  3. 


fha  tolutlona  tM  dlffOrantlAl  eqpMtticm  (6.1)  cc>n«8£»3aidliig  to 
tlMM  ■upla  funetloae  oonstltute  the  enable  functione  of  a  zvMosai  process 
X.  The  proiblSftllltlea  Of  thete  MUBple  functions  can  Im  readily  calculated 
sluoa  the  drlTlns  temus  In  (6.1)  are  gausolan.  In  this  miy  the  random  p?^>ces8 
X  la  rigoroisaly  defined,  aiod  so  is  the  random  procee;^  ^  si-rnr.  ay  (6.1’). 

If  tUa  difference  eq^mtion  (5«1)  is  derlTed  from  the  differential  equation 
(6.1),  these  rendoa  processes  vlll  integer  Talues  of  tim  vlth  the 

random  hy  (5,1-1*). 

fhe  eathenatioal  strvicture  of  the  random  processes  just  defined  is  no 
more  complicated  than  the  structure  of  the  corresponding  random  sequence  t 
ee  batre  ibare.'<y  Introduced  a  ccotlnvums  tine  petraaeter. 

Nov  VB  come  to  a  delicate  ijatter,  the  definition  of  the  gaunslon  vhits 
noise  process.  WO  shall  not  attempt  to  give  a  rigorous  definition  (vttich 
vould  require  adranced  analytical  tools)  out  hope  that  the  foUoving  dieciiB- 
sion  vlll  lend  same  intuit Ive  meaning  to  thiq  important  concept. 

(  Q  )  f  Ol) 

list  T-  -  and  v'^'  be  the  gauasian  rsndom  processes  dafinod  above, 
hut  nov  ve  assuae  that  the  Intervals  over  vhlch  the  sample  functions 
are  constant  are  of  length  q”^  (Vhere  q  ie  a  positive  integer).  Wo  let 
q  oo.  While  this  happens,  ve  must  multiply  the  covariance  matrices  by  q, 
in  order  to  preserve  the  phyelcol  charactorlstlcn  of  these  processes.  Ibis 
can  be  seen  easily  as  foUoira.  Let 

{&.}) 

0  ’ 

this  iB  a  ’-—11 -defined  raiido®  variAble  for  all  q.  The  mean  of  x 
is  aero  because  the  mean  of  is  zero;  vc 

X  i 

var  |jx||  -  E([  /  ] •  -  [  v^'^^(T)dT]) 

0  c 

»  trace  }  J  s(T^*^^(t)T^*^'^  (t)dt  dT 
C  0  ~ 

Ibe  Ijfc*  t  exprasBlon  can  bo  explicitly  eralist^d  fro®  the  definition  of  the 
V  prcx’.Qso);  the  restjlt  Is 
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If  B  !•  oonataat^  this  ttSQiraitiOtt  tonda  to  saro  as  q  »  In  ottier  vorde^ 
if  g  is  Impt  ooastsct  as  q  of  90,  tlwn  the  affect  of  nhite  noise  in  the 
dlftarantial  eq^iitloa 

a*/dt  -  T^^Vt) 

wouli  arantmlly  reduce  to  taro  —  this  is  pb^rsically  absurd.  Heacs  to  kesp 
■mr  Ijxjj^  ■  coast,  as  q  -*  00,  ua  aust  aultlply  the  covBrlancc  wotrlceB  of 
and  bjr  q.  This  aeans  that  the  ■aplltudea  of  the  random  steps 

In  the  saaple  functions  of  and  Increasfi!  as  ‘>/q  ;  on  the  other 

baud,  the  areas  of  the  rvidcai  steps  tend  to  0  since  they  ohaxtee  as 

Vq  •  q“^, 

Guided  by  thsae  obsorratlons,  ws  deflns  the  gaucalap  idULta^oolse  procoan 
y  and  v  as  the  fonaJL  limit  of  and  /<ir 

as  q  -♦  00,  Since  they 

ars  *gaus8lan,  y  ana  v  are  specified  by 

^{▼(t))  -  0,  S(j[(t))  »  0  for  aJLL  t; 

E(v(t)T’(T))  -  6(t  -  'r)§(t),  B(i»(t)x»(T))  «  5(t  o  T)g(t)  for  all  t,  (6.5) 

B(i(t))['(T))  -  &<t  -  T)C(t)  for  all  t,  t,  (6.6) 

uhcre  5  is  that  Dirac  delta  function. 

Bxe  preceding  discussion  shomi  also  that  the  Talues  of  the  saatplje  functionB 
of  T  and  V  are  to  be  regarded  as  delta  functions  of  vanishingly  smaLl  ama,3. 

Mathematically  apeeltlng,  this  definition  Is  of  coureo  sheer  nonsense  aln^ 
b(t)  I0  not  a  vall-dafjjBed  f^sncvion)  It  is  even  more  absurd  to  speak  of  sample 


The  term  "uhxSe”  la  due  to  the  fact  that  these  processes,  like  ordinary 
lljdit,  may  be  thought  of  as  containing  varos  of  every  freq>iency  with  equal  pro¬ 
bability.  Vhen  values  of  v(t)  oocurring  at  dlfferezit  tljsss  axTS  not  iau»SKju- 
dont,  this  Is  no  longer  true  and  then  one  scnaatlines  talks  of  "colored"  nolsa. 


fimotioiiB  ^ose  ’'valuea''  ata  dalta  functions  of  zero  sjraa.  the  Idea 

ef  Tstiite  noise  is  a  very  useful  caie.  Hov  io  this  to  do  reconciled  vita  one's 
aatasBRtloal  conaolon<Mt 

ohR  points  shoulA  bo  es^b&slsod  here.  First,  in  the  usual  applications 
one  noYsr  deals  vlth  the  covariance  matrices  (6.3-^)  directly,  cut  only  in 
conjunction  vith  the  co^grtitatlon  of  integrals.  For  instance,  consider  the 
gausslan  random  process  x  generated  by  (6.1A0,  with  x(t^)  »  0. 

Sine*  2{x(t)}  -  Q,  its  covariance  matrix  is 

ccv  ••  E(x(t)x’(t})  • 

f 

.5y  (U.5  ), 

t 

«  K(  /  «(t,  T)0(T)v(T)dT 

to  “ 

t 

X  /  '<^'(‘f)5,'(T’)**(t,  T')dT') 

tc  ~ 

¥8  proceed  formally,  interchanging  the  evr.ected.-’valuc  opers.ticm  and  nr^eipra- 
tion  with  respect  to  t,  and  using  (6.V)t 

t  t 

"  L  L  ‘f)5(T)ft(T)g'(T»)2'(t,  T')6(t  -  tM. 

Iltilizjb:ig  properties  of  the  b  functlcn,  ve  finally  obtain 

t 

cov  lx(t)]  »  /  •(t,  T)G(T)g(T)0’(T)*'(t,  r)dT  (6.7) 

2ie  derivatlCBi  of  this  result  ia  purely  fonnai,  because  the  rswlcsa  process 
X  haa  not  even  been  defined.  Since  the  sos^le  functions  of  v  end  v 
ore  a&thsoaatloally  nsaningless,  so  is  also  the  differential  equation  (6.1A.). 

But  ve  could  have  obtained  (6*7)  rigorously  by  Ihe  foUowin^  rl^oroiia 
procedure.  Uxe  Integral  (6.7;  certainly  exists  In  relation  to  tSt«  process 
its  limit  q  -*•  oO  also  exist  a,  and  we  can  regard  this  Iliad,  t  ee 
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ccrarisacc  aatrix  of  aam  raujdab  process  not  yet  completely  defined, 

(Jurying  this  method  farther,  ve  can  define  a  randoa  proceoe  hy  specifying 
the  yalxies  of  its  integrals  (iJnecur  functionals)  and  not  assigning  any 
neenlng  to  Its  aaiqplB  functions.  !Qbils  Idea  iias  developed  by  Wiener  in  the 
1920*8  and  still  constitvites  one  or  the  main  tools  of  the  rigorous  theory 
of  random  processes. 

Die  second  point  Is  this.  !BSe  subterfuge  of  dealing  only  «ith  r 

of  a  random  process  is  not  really  satisfactory  bocaujst.  no  maaning  is  attached 
to  the  differential  equation  -l)  Itself,  In  recent  years,  a  new  approach 
has  evolved  lAilch  Is  relatively  free  of  dlfflcultleB  of  this  sort.  Cie 
imiteHnolso  process  Is  regarded  as  a  gasarallzed  random  process  which  la 
the  zmndom  counterpart  of  the  concept  of  a  gsperallzed  function  (or  dlstributign ) 
Invented  by  Sobolev  and  L,  Schwartz.  %1b  technique  i  r.  used  successfully 
by  the  Russian  school  led  by  I.  M.  Oel'fand  [16,  I7]. 

As  mentioned  In  the  Introduction,  as  far  as  the  present  paper  la  concerned, 
Ve  regexd  the  difficulties  stirrounding  the  rigorous  definition  of  random 
processes  as  purely  tecfanicalj  we  shall  not  hesitate  therefore  to  take  llmitc 
foxnaUy,  interchange  the  expected  value  operation  and  integratloa  with  rms- 
poet  to  tine,  etc.  (nie  reader  will  noiilco  that  the  "inadialeslble  b  xpo*'  are 
used  only  to  derive  Integrals  of  the  type  (6,7  )  —  these  results  covOd  br 
rigorously  Justified  by  Vlener's  technique.)  We  shall  devote  a  future  paper 
to  such  problems,  using  the  Oel'fhnd  theory. 

Ibe  definition  of  a  random  process  by  means  of  a  linear  dynamical  system 
eoccited  by  white  noise  viiw  esqphaaized  in  the  engineering  literature  particul¬ 
arly  ‘V  Beds  and  ShaimoD  [18I  and  Zadeh  and  ftsgasztni  [I9].  Kot  only  is  this 
assxnqptloa  physically  pleasing  but  It  lead.*  to  a  cloax  and  ccnvenlect  aathe- 
mtlcal  frame  work. 


•fe  imaglse  that  nclm  ojflglmtas  c*»  a  microscopic  Isrfel.  MacrosKX)pic  siotse 
Is  clearly  gausslan,  because  of  the  fivqparpoalticn  edf  sasy  saaLll  randosi  effects  a 
and  It  is  white,  because  the  dynanlcs  ef  mierosoeplo  phenonena  axe  very  lUat 
on  the  tlms-eeale  of  the  sdcxosGoplo  observer.  AFProolaMe  dynamic*!  ts 
oQsm  abomt  coly  on  a  wsuuseoostfie  realm  and  ara  roprosanted  by  (4.1). 
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7*  AxlOBatlc  definition  of  tbe  aauaB-markcnr  sequence  am  proceea. 

Dia  deflnltioat  glvan  In  tbe  preriouB  aaetlona  may  aeem  to  be  highly  arhlt- 
ary;  guided  "by  phyalcal  Intuition,  we  postulate  a  "BechaniBm"  la  eimply  t 
natxxral  repreaentntion  of  the  process;  ve  can  derive  this  representation  by 
taking  the  gaussian  and  markovlan  Properties  as  fundamental  axioms.  In  other 
words,  from  a  logical  point  of  view  there  is  no  loss  in  generality  In  starting 
with  (5. 1-1*)  or  (6.1-1*)  as  the  basic  definition. 

_  f 

AXIOMATIC  DEPIltITIOK.  We  say  that  a  l-parameter  family  of  randOin  vectors 
.i{^)  (■^  ■  integer)  Is  a  gjausa-aarkov  aeguenee  if  it  has  the  following  proper¬ 
ties: 

(I)  the  sequence  is  gaussian;  that  is  to  say,  for  any  fired  integers 

t,  T  the  random  vectors  x^t),  ^(t)  have  a  Joint  gaussian  distribution 

with  mean  ii(t),  y[(t)  and  cross -covariance  matrix  L(t,  t), 

(II)  the  sequence  is  markovlan,  in  other  words,  jPor  any  Integer  >  t^ 
the  strict  narkcr/lan  property  (5.6)  is  satisfied. 

Similarly,  we  say  that  the  l-parameter  family  of  random  vectors  x(t) 

(t  IB  real  number)  is  a  gauss -markov  process,  if  the  preceding  properties 
hold  with  t,  t  being  real,  numbers,  and  if  L{t,  t)  is  nonsixigular  while 
E(t,  t)  is  a  continuously  differentiable  function  of  t,  t.  (End  of  defini¬ 
tion.  ) 

First  we  shall  study  random  sequences.  We  asBumc  for  the  LicKnent  that 
li(t)  =  ^.  Wo  let  t  >  T  be  Integers  emd  write 

®(t.  r)  =  E(t,  T)E*^(t,  i)  (7.1) 

By  (5.11),  v(t)  ^{t  +  1)  -  E{x(t  +  l)!x(t)3,  is  a  gausolan  nmaom  vector 
with  zero  mean  which  lo  ifence  can 

x(t  +  1)  ^(t  +  1,  t)x(t)  +  «(t)  (7.?.) 

which  Is  Identical  with  (5-l),  except  that  F  I. 
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Morsove?,  v  In  (T.2)  ii  a  ^its-^.oise  ee^uoncB,  For 

E{x(t  l)  IxCt),  x(t  “  l))  -  E(x(t  +  1),  w(t  -  l)3j 
by  dsflnltlon,  x(t  -  1)  and  v(t  -1)  are  indepettdent  of  each  other,  bo  that 

«  E(2{t  +  l){5(t  -  1)J  +  E{x(t  +  l)|w(t  -  D) 

«  ®(t  +  1,  t  -  l)x(t  -  1)  t  E{^(t  +  1,  t).r(t)lx(t  -  1)) 

+  Efv{t)lxit  -  D); 

f 

by  (7.2)  the  middle  term  In  the  prceedlng  equation  ie  Just  «»(t  +  1,  t)w(t  -  1). 
Hence  ■ 

E{x(t  l)!xCt),  x(t  -  1))  =  i(t  +  1,  t)x(t)  +  E{w(t)lw(t  -  1)} 

By  the  narkovlan  property,  this  la  also  equal  to 

«  ' 

»  +  l)|x(t)) 

Which  iaqplieB  that 

E(x(t) lx(t  -  1))  «  0 

proving  that  w(t)  and  w(t  -  l)  (being  gaussian)  are  mutiially  indapexidenfc . 

J.-  Ui.  ULtiCL  f 

S[v(t)|x(t  -  1))  -  E(x(t  +  1)  -  i(t  +  1,  t)x(t)lx(t  -  1)} 

-  [i(t  +  1,  t  -  1)  -  iCt  +  1,  t)-i(t,  t  ~  l)}x(t  -  i) 

BO  that  the  independence  of  w(t)  and  x(t  -•  l)  will  follow  If  we  ptovt  the 
identity 
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In  flact,  let  3c(tj^)  be  eurbltraiy  and  consider 

by  an  eleisentaxy  property  of  conditlonfil  expectaticna  [iPj  p.  55 

-  E(E{»(t5)|x(t2),  x(t^))jxCt^)), 

Ui^ing  the  soarkw  property, 

-  ElE(i(t5)ix(t2)}ix(t^)}; 

using  gauaalannesB,  we  calculate  the  condlticaaeil  expectation  by  (B.6)  and 
*  write  by  (7.1), 


irtiich  proves  (7.2)  since  x(t^)  vae  aiijitrary. 

Extending  these  argmeents  by  induction.  It  follows  easily  that 


cov[v(t). 

x(t)  ]  r>  0 

11  t  £  t 

covlv(t). 

v(t)]  «  g 

if  t  /  T 

(7,^-m-)  show  tha.u  Ipuc  recursion  relation  VY.2)  uexines  a 
gaussHBorkav  sequence,  which  has  zero  oicitn  and  the  prescribed  covariancr;  uifttrls. 
f)  (provided  we  let  x(t^)  be  a  gausnlan  random  vector  with 

i(to)  -  £(V 


Bie  coTariance  ma’crlx  of  w  la  normegatlTe  definite  but  may  be  ulngu-ljir. 
For  conrenlence,  ve  factor  It  into  the  form  r  T’  and  tiien  (T.2)  Deccsob 

x(t  +  1)  -  E(t  +  1,  t)x(t)  t  T^(t  t  1,  t)v(t),  (7.5) 

vhera  ir  is  nov  defined  as  an  m-reotor  ahlte-noise  sequence  with  zero  a»an 
and  t»it  Tai’ lanes. 

Hcnr  we  can  remove  the  soro  mean  assumption  by  considoring  x(t)  -  jji(t) 

•  -e  .-et  ..f.V,  #*•«■  ff\*. 

Xli»uWUJL  Ox  ^  I  a^/ 

x{t  +  1)  «  «(t  +  i,  t)x(t)  +  r(t  +  1,  t)v(t)  +  d(t), 

! 

where  the  deterministic  coiqponent  d(t)  is  defined  as 
d(t)  -  y  1)  -  ♦(!  1,  t)ii(t). 

How  let  z  be  a  gausslan  random  ee<iuence  which  1b  causAlly  related  to 
x(t),  that  is  to  say^  the  conditional  distribution  of  z(t)  giyen  x(t) 
is* identical  with  the  conditional  distribution  of  z(t)  given  x(t}^  z(t  -  l), 
^t  -  1),  z(t  -  2),  .  Define 

«  H(t)x(t), 

T(t)  -  z(t)  -  H(t)x(t). 

Proeeedlng  as  before,  it  can  be  proved  that  v  is  s  gauBol;rm  wbite-noiae 
sequence  (possibly  correlated  with  w(t)). 

we  have  now  proved  that  (5^1-1*)  Is  a  representation  of  the  abstractly 
defined  gausslfin  sequences  x  and  a.  3iJ.a  repr^sert^^tion  Is  oltearV: 
seide  from  the  uciieq>ort«nt  arbltrarlnoB!i  in  defining  r  and  beatca  a3^o  w. 

SSie  derivation  of  the  repieaentation  (6.1)  proceeds  similarly.  Uiere 
p^-'nts  require  howerer. 

Since  aeeoaiel  to  be  ncnsingular,  it  followo  from  (7.I) 

that 

♦(t,  t)  ■  S  for  all  t. 
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If  E(t,  t)  la  continuously  dlfforontlaible  in  t  for  all  r,  then 


y(t)  =  Si(v,  «  t  * 

T  ->  t 


®ft  r  h-  tl  I 
cl: - ^ - <=  ®lt  ,  T) 


la  defined.  Therefore  by  (7.3)  iCt,  t)  satiafieB  the  differential  equation 


dfl/dt  »  Zi^)t  for  all  t  h  t. 

By  (^'T)>  £  will  than  aatiefy  (7*5)  for  all  real  numberG  t^,  t,. 

f 

On  the  other  hand,  any  acalar  of  the  form 
<p(t,  t)  -  exp[a(t)  -  a(T)] 

aatiafiea  (7.3)j  but  if  ^  is  not  differentiable,  we  cannot  regard  It  aa 
the  aolutlon  of  a  differential  equation,  so  that  the  representation  (6.1) 
cannot  exist. 

if  the  gausa-aarkov  process  lo  stationary,  l.e.,  E(t,  'i )  depends  only 
on  t  -  T,  then  it  suffices  to  assxuac  that  £  Is  ctmtinuoiu!  in  t  at 
t  «  T  »  0  I2O,  p.  46]  fco  assure  that  «(t,  t)  Ib  the  tronaition  matrix  oi 
a  differentiiil  equation,  and  thus  to  prorve  the  representation  (6.I). 

Slmilair  results  were  obtained  a  long  time  ^^f^o  by  Doob  [21]  and  Wang 
and  Uhlenbeck  [22],  but  the  present  derivation  la  alwpler. 


«• 

Incidentally,  neither  Doob  nor  Wang  and  Uhlenbeck  make  any  aesumptlorui 
about  continuity  of  £.  Without  soma  such  assumption  their  results  are  in¬ 
correct.  For  instance,  assuming  statlonarlty  cr(t,  t)  «  1,  ond  considering 
the  scalar  case,  (7.3)  reduces  to  o(tj  -  t ^  )  o(t^  -  -  t^).  Moreover, 

since  D  IS  a  correlation  coefficient,  jo}  a  1  euia  o  Is  an  even  I'unctljn 
of  its  argument,  Doob  [22]  asserts  that  the  only  non^er-o  ikmctlon  o(t)  satiafy 
ing  these  conditions  is  o(t)  =  expf-a|tjl,.  whej^e  a  1;-  u  nomwgatlTc  consttvn 
But  this  is  false,  for  one  can  construct  —  using  the  axiom  of  choice  —  fUnc- 
blona  which  satisfy  these  requiromeatB  but  are  everywhon^  diBcontinuouo  so  that 
they  caroiot  be  represeinted  by  an  exponential. 


A  ewple  pyMotlcn  probXwiu  Bafore  enflaiurking  on  the  detailed  and 
vffitt^oidably  eosfplex  •tudjp  of  prediction  and  fllterjjag  in  the  general  ease, 

It  will  be  helpful  to  pause  for  a  minute  and  solve  a  sls^le  problem*  Consider 
a  gausBHDarkov  prooaes  generated  by 

-  -  OT.^  *  kg  \ 

dxydt  ■  -  OKg  +  v^{t)  ^  (g  >  O)  (8-l) 

I 

f 

nve  ordinary  block  diagram  of  the  system  is  ahown  in  Fig,  J4A.  Koto  that  the 
output  of  the  system  con  be  observed  vithout  any  aorr\q?tlng  noise. 

Die  matrices  F>  0,  H  can  be  reed  off  by  inspection  from  Fig.  Ua.  They 

are: 


We  wish  to  esti^Mte  the  value  of  "*■  9)/  vhere  9  >  Q,  !iavlr.g 

obsorvod  all  past  outputs  of  the  system  up  to  time  t.  This  probi’:® 

Is  ldentlc8Ll  with  one  stated  (in  different  language)  In  [  5,  p.  mOo].  We  shall 
solve  this  problem  hers  aiter  very  few  preliminaries  and  in  a  very  raucli  slmpier 
way  than  in  [  j],  whex-e  the  EOiuticai,  using  older  mcthcxlB,  appears  after  wO 
pages  of  preparation, 

hy  linoai-lty,  the  qxiantlty  x^(t  +  9)  depends  on  two  things:  (i)  the 
state  x(t}  end  (ii)  the  excitation  w^(-r)  In  the  interval  [t,  t,  +  5], 

Since  ifl  a  «faite-noi,8e  process,  its  future  values  cannot  be  estimated  in 

any  way  from  post  observations;  oi',  uioix  precisely,  the  best  catlioatc  io 
simply  the  mean,  which  in  this  case  is  zero.  Expressing  this  in  writing,  we 
have; 
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(8.3) 


MXy(t  +  0)j*^(T),  T  <  t)  -  H  4(t  +  d.  t)S(^t)!z^(T),  T  s  t} 

t+© 

+  E(H  /  «(t  +  ©,  T)a  w(T)dT|z  (t),  T  S  t) 

"  t  “ 

«  H  ♦Cfc  +  ©,  t)®(x(t))z^(T),T  S  t) 


lenr  iw  MdeulAte  the  tranaltlon  Matrix  of  (8.1).  BjIb  is  easily  dene 
hy  noting  tbut  -  reeponae  observed  at  the  1-th  Integrator  in 

at  tlae  t  tf  a  rirlt  issrilss  Is  applisd  to  ths  input  of  tho  i-th 
lategstituf  at  tine  t  <  t.  ©la  reaxxlt  Iss 


l(t,  t) 


“  “f)  (t  -  T)c“^^^  ' 

n  .-«(t  -  t) 


(8.4) 


What  la  the  cocditlcnal  expectation  of  x(t),  giren  aU.  the  obaerratlona 
*  '’P  to  time  tT  Clearly,  x^(t)  la  knoim  exactly  bec&uee  the  obaerm- 

tloas  are  not  corrupted  by  nolae.  On  the  other  hand,  by  (8.I), 

X2(t)  -  ax^(t)/dt  +  CBc^(t)  «  dx^(t)/dt  +  C»^(t)  (8.3) 

Bet  the  vhlte  nolae  process  psissoB  .  through  tsc  '’Baoothlng"  operations 

as  shown  In  Fig.  H.  ©us  Xg  la  "sExxyther"  than  (Xg  is  the  so-c&lled. 

Omateln-Bhlenbock  process  [211),  and  x,  la  swoother  than  3  'n  p'  rticul-srj 

^  i  ^  ' 

the  x^  procesa  has  a  derivative,  -oes^  +  Xg,  which  la  a  well-doflne<5  rmiCcm 
process.  Hence  we  nay  evaluate  -the  rlj^t>hand  side  of  (6.3).  "Htua  by 

(8.9) 

A 

lifxi’t'l  tK(T),  T  *  t)  “  Jt(tlt)  «  t 

Ctt,(t)  +  ds, (t)/4t 

and,  using  (8.b), 
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K{x^(t  +  9)|£(t),  t  i  t}  » 

«  ^(t  +  Qjt)  "  e'^[(l  +  c<9)Zj,{t)  +  eda^(t)/dt] 

fflils  agre«B  irlth  [J,  p.  eq.  751*  optliMvl  px’edlei.cr  Is  shown 

in  irig.  4S.  Hm»  ;;7/jahol  n  denotes  A.ij*fer«ttitlatic>n  with  respect  to  tinse. 

A  moat  Interesting  rsftture  of  this  result  Is  that  It  le  independent  of 
the  Tarlance  of  Using  the  concept  of  xdilte  noise  one  can  almost  com¬ 

pletely  dispense  irlth  the  machinery  of  probability  tbeory  to  get  the  anawr. 

Another  important  point  Is  the  fact  that  the  optimal  prediction  In- 
TOlves  the  operation  of  differentiation.  TBila  operation  is  not  resillEftLle 
In  practice!  nathatsatlcSLlly,  baoause  differentiation  is  an  unbovnded  operator; 
and  physically,  because  the  Ideal  diffeirentiator  hee  infinite  bondwlth.  We 
shall  see  later  that  this  unpleasant  feature  of  optlmai  pi*ed,ictlon  lo  a 
consequence  of  the  assunrotlon  that  the  outimt  of  the  system  (8.1)  can  be 
observed  <jS£Ctly.  If  we  Introduce  white  noise  in  the  observat:'  -ns,  with  no 
matter  how  llttlo  energy  per  unit  time,  the  difficulty  dlBapiware.  We  Luve 
therefore  two  choices  in  formulating  a  prediction  or  filtering  problem  in 
continuous  time ; 

(I)  either  we  assume  that  the  observatic-ns  on  the  random  process  can 
be  nade  with  infinite  accuracy  —  then  we  must  approximate  tiie  ideal  pre¬ 
dictor  which  la  not  physically  realizable; 

(II)  or  we  ansume  that  thfc  ousenratlous  are  contamJjmted  with  idilfce 
noise  '  then  the  optimal  precllctcr  in  aiwuys  rcai i iJiblc. 

Wb  ahall  ai.vnsiyB  ofaoo«e  t-he  second  cscisuption  which 
from  the  phy-lea;.  |/oint  of  view. 

It  Is  clear  that  this  difficulty  does  not  arlH«  in  discrete -time  bystesu) 
and  therefore  the  question  of  whether  or  not  the  observations  are  exact  is 
tanaterlal. 


iJtO 
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9.  atatMtaet  and  exMBPlaa  ot  thm  flvt-»£3^^&_2.r^Xga«  Vs  hare  no^ 
ft?rlY«d  at  the  nain  pert  of  the  mtorisl.  For  eaae  of  reference  ee  restate 
the  escsnce  of  the  dlscueeion  In  8eota.  £,  5-T  eo  foUcnrB: 

FIiaBREK}  PHOBUSM.  Conwlder  the  aange^markoT  seq^once 

r(t  +  1)  «  i(t  +  1,  t)x(t)  +  i:(t  +  1,  t)w(t);  (I^) 

a(t)  »  HOt)i(t)  +  x(t) 


vaere  v,  v  are  ’tfelte-aolse  3e<;.»^neeo. 

Or  confilder  the  aeauag^iaarhcry  ptogpes 

d^dt  =  F(t)x  +  0(t)v(t),  U-i 

z(t)  =  H(t)x  +  v(t), 

where  ▼,  w  are  ftaufielan  white -notee  proceBseft. 

In  either  ceae,  ▼  H  are  explicitly  dcflnca  •feJar.I.c^n 


Elv(t)v(T))  -  5(t  -  T)R(t),  E{v(t)w’(ic))  =  5(t  -  T)3(t),  Vov_;^  t,  s 


E{v(t)v'(T)}  ^  ft(t  -  T)C(t)  for  hH  t. 

fir,  these  expreaslone  t,  t  are  Integers  reep.  real  nvaabg^s;  - 

tte  kronecker  delta  reep.  the  Dlrar  delta  function. ) 

How  Boppose  the  -ivctual  vaJnee  of  tha  randc*.B  zf-r )  ‘~5>- 

oLaorvecl  jxi  tbc  Intorral  *  T  s  t. 

is  the  candltional  probability  dlgtrlhutlgi^gf  ^(t^; f 
We  Bhall  refer  to  (l)  ao  the  aodel  of  the  orocees- 

tcirnirology  ie  taotf^ated  "by  ocssaunlcatlon  theory:  one  raty  regard  in’. 


Ip 


.ult  Ive 


V 


x(t)  “  F(t)x(t) 


l4l 


^  mnum*  t  1b  tha  noltc.  s  is  the  elaasd  (message  plus  noise); 
V  Is  the  reason  ehy  is  a  raadon  rariahle. 

It  vlU  te  conrealence  to  use  from  nov  on  cerrain.  spociai  notaticair.. 
Let 


be  the  ocoditlonal  mean  of  x(t^)  giron  ohBsrred  Taluea  of  2(7)  for 
t  s  T  <  t.  Similarly,  lot 

x(t^|t)  -  x(t^)  -  x(t,  jt) 

f 

be  the  "error"  betvoen  the  actual  ^ralu6  of  x(t,)  and  Its  conditional  expec- 
tatioo.  Wo  note,  by  (B-ll),  that  x(t^|t)  and  x(t^jb)  are  independent 
random  variables.  Finally,  let  the  conditional  coviiriance  jjatrlx  of 
x(t^|t)  be 


E(t^!t)  -  E{x(t^lt)x’(t^|t)) 


The  (jiiantltlea  v(t^|t),  v(t^|t), 


etc.  are  defined  sludlarly. 


By  j^VssljUQaesiB,  the  aoliitlOK  of  the  flltcrlni/.  probleiu  ^  .  .^a-ol valent 
Tio  cocajutiint  xyt,  ib)  fetal  Z(tj^jt), 

?ery  mssy  different  problems  ore  included  in  the  matrix  equal  iosio  fl  ) 

(x^). 


(9.x)  EXAMPLE:  Qynamleal  systems  snb.iect  to  mndcro  d jetvxffaances  end 
meaBureaK-nt  noise.  Consider  a  physical  dynamical  ayetem  (an  airplane,  apaco 
vehicle,  or  chemical  plant).  Assume  the  system  is  liiicox-.  Ci-;  autbe  of  the 


ayotem  cojinot  be  obserrou  dli-octlj'  Trxt  crly  thi*ough  the  output  y(t),  vrhJch 


con  be  tBeagvsred  only  in  the  presence  of  additive  gausaion  noise  v.  In  addl- 
tjon,  the  aystism  Is  subject  to  random  dlstiarbances  (atmospheric  turbulence , 
meteorites,  chemical  impurities)  in  the  form  of  the  gausBian  white -noise  pro¬ 
cess  V.  The  equations  of  motion  of  the  system  are  evidently  (5«1)  (^-X), 

provided  we  add  a  deterministic  forcing  term  u(t)  to  (5.I)  f*^d  (6.1)  to 
nc,co\int  foi'  control  variables  (rudder,  control  Jets,  catalystp).  In  order'  to 
control  the  ayetem,  It  la  necessary  to  know  the  state  varJables.  They  car.  be 
"reconstxajcted"  by  means  of  an  cprimal  filter,  Bie  variance  of  v  and  w 
can  often  be  specified  (within  an  order  of  magnltijsde)  by  phyelcaj.  con^iidejrati-ono- 
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(9-2)  E3MMPLB,  It  la  act  neceasary  to  aas-une  in  the  preceding  eaangjle 
tliat  the  ineasurement  noise  or  the  randon  dlsturhances  are  vhlte.  One  can 

alimys  represent  correlatioa  by  adding  wore  state  variables.  For  instanco, 
the  InstniMDts  idalch  neasvoe  3[(t)  nsay  have  dynaadcs  of  their  own  and  the 
noise  nay  enter  at  the  output  as  well  as  the  Input  of  the  institnneiitB.  8oe 
Sxatqple  (llt>.^)  for  a  problem  of  this  sort.  After  the  additional  <!yneicd.cal 
effects  have  been  taken  into  account^  the  describing  equations  cen  always  be 
reduced  again  to  the  standard  form,  perhaps  after  seme  radefinition  of  vari¬ 
ables. 

(9*5)  XWtPLS;  letination  of  rjaraaetera.  Biis  la  a  very  coaeaon  prcbJjeT?. 
in  atatlstice  [23>  Chapters  52-3^1.  Suppose  we  are  given  a  feaiily  of  fmetloiit' 
i®l,  ...,  m  and  J  »■  Ij  n.  Wa  can  asasure  m  randm  varl.'i'^lco 

a 

+  v^(t)  (i  -  1,  a)  (9.^) 


In  the  presence  of  gsusslan  white  noise  v.  (t  ) .  Sic  problem  is  to  form  the 
'’best  possible  estimate"  g  of  Q  based  on  observations  of  ir;  -•rewe 

intsrjal  ft^,  t], 

>*e  cap  easily  reduce  the  problem  to  the  context  of  { 1 .  )  or  ( T  ^  )  m 

c 

foUovs.  Let  V  «  0,  Vs  can  then  regard  0  as  the  unkno«m  state  x(t)  of 
(  j^  )  or  (l^)f  provided  that  we  tan  represent  the  functions  as 


where  t)  are  eleMenta  of  a  transition  seitrix; 


poses  a  reatrlotion  on  the  adalSBlble  funotloas 


Of  course,  (9*^)  1®- 
but  since  any  cs;ntlntu>ve 


function  oaa  be  approoclmated  by  solutlccs  of  ordii^axy  diffej-eatia.l  eqyaticaie, 
this  is  not  a  ^'leriOKie  llJBitaticn  in  practice. 

Intuitively,  this  problem  cexx  be  risua^llscd  a«  geaseralir-ed  curve  fitting. 
Ws  havo  w  sopeilmental  cvrreti  reiTCseated  hy  'Btlueo  of  the  rsasdoai  varlabls 
nuase  experlaontal  curves  to  be  fitted  9laul.t>an«ously  by  lic«ear 
caBblnatioms  of  tsKooth  curves  vw»«  fbailly 

?S»ts  prcblaa  css:  be  solvod  easily  cron  without  the  eems^tlG-  tiaet 
1«  a  wfeiteHQOiae  prowSii.  Bee  Sect.  15  . 

(9-6)  IXiMFLi!  Ooiaiiwnl nation  sarste^  A  rexy  elerssataxy  tsodal  of  e. 
coaganDlcatiuu  system  might  be  the  following.  A  mssattae  is  e  ssEsple 


of  the  ttmdosa  precoBB  defined  orer  sooe  Interval,  eay^  t^].  ihe 

tri&8!Alttod  asBBajSB  j;  i®  contaninated  "by  nolee  v  before  it  reachaa  th;; 
receiver.  ’Scm  teatbeiaailcai  problem  in  receiver  desigsi  is  the  following. 

Olven  the  obeerred  Tal^iefl  of  *(t)  on  the  interval  t),  what  in  the 

beet  estlmte  of  that  saagple  function  of  the  £  process  which  act'oalLy 
fjccurradT  In  other  worda.  find 

v{"rjt)  for  all  t  in  [t  ^ 

Bil8  problan  ia  quite  difficult  becattao  It  might  involve  slaultaneoualy  both 
prediction  end  smoothing;  no  adequate  solution  exists  as  yet  iii  the  fraiaDwork 
of  this  paper. 

It  should  be  noted  that  this  la  also  an  estlaatlon  problem.  Unlike 
Example  (9.3)  TAere  the  xaiknorTO  paraaneter  wan  a  (finitc-dimenDlonal)  vector, 
here  the  unknown  parameter  Is  a  more  ccsagplex  msthcaatlcai  object  --  a  reai- 
,  valued  function, 

5Bse  formulation  of  the  filtering  problem  given  here  is  different  from 
the  comrentlanal  formulation  ■  in  the  engineering  literature  [1-5].  The  two 
points  of  view  can  i-  eacllj'  reconciled  as  was  mentioned  briefly  Jn  Sect.  P. 
and  as  is  discusced  In  moro  detail  in  ttw  next  section. 


ID.  -fipfr— iiAtlopB  of  tb»  filtering  turoblem.  Go*m«otlaaj» 

our  vurAlcu  of  tbe  fllterl&c  pro1)I«i  ftad  othor  points  of  rlav  appesred 
in  39ct.  2.  To  aid  the  render  in  certain  appUe&tions  of  the  thaexy,  ea 
•waarlBe  here  eowa  well-toio«u  ftaots. 

Often  ^le  filtering  prohlem  le  foraulated  ea  foUon**.  Find  an  eatlMate 
t‘(\)  of  x(t^),  haoed  on  ohaervationa  of  b(t),  t^  a  t  s  t,  idileh 
islntBlEee  the  ennected  Xoaa 

E{l{LCx(tj^)  -  x»ft.jL))|*(*rO,  t^  S  T  a  t}}e  (lO.i) 

iSxe  loss  function  L  la  deflxied  an  tollowo.  Let  p(x)  be  a  real-valued 
nonnegatlTO,  convex  function  of  x! 

+  (l  -  X)^)  <  Xp(x)  +  (l  -  X)p(j),  where  0  S  X  S  1. 

Tben  L  la  a  real -valued  function  of  x  auch  that 

L(0)  -  0, 

L^Xg)  >  ^  «5en  0(^2)  p(x^)  *  0-  (10,2) 

Evidently  p(x)  measures  the  distance  of  x  freen  the  origin,  end  tiie  lost: 
is  nondecreaelng  with  this  distance.  Obs^rre  thc«,t.  L  need  not  be  eusi^ps . 

The  solution  of  the  preceding  problem  is  contained  in  the  theorem: 

(10. 5)  Ipt  X  be  an  n-dlaenaional  randem  vector  with  mean  y.  ton  cl 
dlatrlbutlop  runctioit  ^(i}* 

^  1^  *3YTey*t''^j  cal,  wbovt  **nd 

(B)  r  la^mlBodal  (l.c.,  convex  for  s  1  =  1,  n), 

T*-,,..  E(L(x  Ip  BlnlalEed  by  setting  x  ■* 

For  the  proof,  see  Hherman  [  2h  ], 

The  preceding  conditions  are  obviously  satisfied  by  a  ^vinHjj'.n  distribu- 
tlcn.  Hence  (lO.l)  is  minimized  by  taking  for  ^(t^)  the  condltiunal 
expsc^sftion 


-  S(\b)  ■  S  T  <  t), 

vhoM  caleul&tico  1b  a  part  of  tha  filtering  problam,  as  stated  in  Sect,  ?. 

A  spticlnl  lo»B  function  la  Ii(x)  «  fe!!^  ^ihere  P  is  nonnegntive 
definite  in  thlfi  caBCj,  (10- j)  1*?  truf>  vlthowt  any  aaovarotion  ot  tbs  bistrt- 
butlon  function: 


»(ll5  -  2^llp)  "  «(yip}  + 

-  cotuit,  +  ■*■  1 

-  Ife*  -  lidp  +  const. , 


uhlch  Is  obviously  minljnlzed  again  ®  ^  It  should  noted  tU  xt  titiJlL 

«  ' 

rsBult  doea  not  depend  In  any  nay  on  P.  In  particular,  Buppose  P  “  aa •. 

Then  the  best  eatlMte  of  a'x  In  a'||. 


In  the  literature  one  often  reade  aoap  Judgnaents  to  the  eftoct  that 


only  fl<pvBred  looB  funczlonB  can  be  treated.  Ihlei  is  incorrect  or  j^t  iKaat 
mldJjeadlng,  Bue  preceding  dlBCUBSion  shovs  that  she  condilional  tncan  Bupplic^ 
the  minllBUBi  expected  .1oh«  for  aaoy  loss  functione.  IHtos  the  Ioah  function 
ployB  a  aecondary  role.  Of  ccmr*e  if  the  condltlcaial  dietrlbutdon  Is  isnfTwn, 
the  beat  estljaato  ^  can  be  coa^puted  for  any  Iobb  function. 

Ftnaliy,  Me  aay  vlRh  to  find  the  boat  eBtlnate  3^  vhleh  1»  a  r*  Inoax 
function  or  the  data  z(t),  t  a;  t  £  t.  We  have  Been  th&t  for  a  ''rcascruible " 

— '  r  0 

loos  function  the  beat  eetlBBts  x*(t^)  in  the  jifusslan  crcc  i<?  the 


.  •*  wli* 


C<jnu,x  UrOflAal  SA^'OCv^UXOn  x(^/^|ty,  Mxiich  xi>,  vVi  V -i..  T 

z(t).  The  calcnJation  of  this  eatismte  involves  only  the  iseaii*  luw.  cuvtir '  ujicn 


c-l  -l.v.  gnunaldm  pidcobb,  Thua  (aa  ve  have  pointed  out-  already  in 
x^(t^)  »  x(t^lt)  is  clearly  the  beet  llueor  eatlnwte  for  the  C.IO.01'  of  al  l 
rac.dota  procesBeu  vl  th  the  cesa  uveeua  and  covariance  usatrLcee  an  the?  gaurie  i  txn 
proceaa  for  vhlcli  x(t^jt)  woe  cooputod. 


have  now  proved; 


(3X).4)  J£  a  llpw*r  eatiaate  Ib  optl^X  for  one  xose  functlcn 

of  type  (jr>,c:),  It  Ic  optla&l  for  aJJ  such  loaa  functlong. 

Hence  tha  linear  mlninwa  o»an  sqw».re  l":'V  if;  oprtlassJ.  for  all  loss 
jCMnctionfi  (10.2). 


11*  aolutil’-Tn  of  tiae  riltarlag  pi-otlea  for  rapdcB  seguencea.  Accordine 
to  tho  problem  BtKt«Heut  in  Oeot,  $j  ve  are  to  cora^mte  the  condltloml  distri¬ 
bution  of  ^t^),  given  observations  z(t)  in  the  interval  £  t  £  t.  By 
gausslanness,  this  1b  of  course  equivalent  to  ccsHputing  conditional  meonB  and 
co»'ariaoce  auatricea , 

It  Trill  be  convenient  to  work  In  terms  of  '^t  +  l|t)  end  ^(t  +  l|t)- 
Ftrst  we  show  bow  to  reduce  ibc  probie-?  to  the  cc^putaticr.  of  thccc 

qunntltlea. 

Let  t,  *  t  +  2.  £ty  repeated  use  of  the  de^lnJ.tvK  fqratlon  (l,,'  of  a 

tdm  Ca 

random  sequestcoj  we  obtain  the  expression 

t  -1  ' 

X 

x(t-)  -  f(t^,  t  +  l)x(t  t  1)  E  t(t  ,  T  +  2)i<t  +  1,  tMt),  (ll.l) 

.  T-T+1 

wbicb  is  valid  for  aJl  t^  k  t  +  2.  Taking  conditional  expectatlonc  of  both 
a Idea  with  respect  to  l(t),  ve  obtain  the  relation 

‘  z(t^jt)  -  i  *■  i)»(t  •  l|t)  when  k  t  +  1, 

using  the  fact  that  w(t  +  l),  v(t  +  2),  ...  have  aero  iaean  and  aa-c  inder->,id- 
ent  of  s(t^)j  •••!  j5(t).  We  see  that  le  obtalaed  by  cxirapolatintf 

x(t  l|t)  by  Keens  of  the  trensltlon  ■atrlr  of  th*  randcai  (i^)- 

let  =  t.  Taking  condltloeai  eaipec  tat  ions  on  both  alcLca  or  (i^)/  -’c 

get 

x(t  Ijt)  -  1,  t)x(tjt)  l(t  ♦  1,  t)v(tjt).  ( Ll,;>) 

It  is  easy  to  see  that  w(t)  is  indaprwflsnt  of  e(t),  s(t  -  1),  since 

F(w(t)}  »  Oj  It  follows  by  (b-6)  that  w(tjt)  -  AXt)&(t),  vhen-  - 

5(t)cor[v(t) U  Since  we  hjsfve  assumed  (see  Sect.  that  *(t  +  1,  t)  ii-  .<or- 
sliisular^  (11.2)  can  be  solved  for  x(t[t),  in  ~fch«r  word.*^  ye  c»io 
as  a  linear  Ibaactlon  of  Jt(t  +  lit)  and 
If  j^ttl,  t)  is  singular,  this  procedure  fails  mnA  we  saiat  work  v'th 
x(t|t)  instead  of  ^t  +  ijt)  as  the  basic  quantity  -  thfi  roqiilred.  aocific:^- 
tions  are  easy  bat  tbe  resullant  foxiexlae  are  leas  alnple. 


Xf  t  <  t,  «s  cannot  ssprees  x(t.  |t)  Bolaly  in  terms  of  x(t  h  ?i.\t) 
and  £,(t).  AS  a  cjatter  of  fact,  x(t- |t)  will  "be  in  general  a  linear  combina- 
tlon  of  ii(t  +  l|t),  r(t),  x(t,  +  ijt),  fortimately,  fchia  la 

seldca  re<xuixed  in  practice.  Ibe  dst&ila  are  ineaBy,  and  ve  omit  them. 

Bio  computation  of  E(t^ft)  la  BiAllar.  Since  the  explicit  expresjyions 
for  £(t^jt)  will  not  bo  noodod  in  the  seijual,  the  detallB  are  a@Rln  omitted. 

Bxo  reaalnder  of  thla  jMsction  la  concerned  prlrearlly  vlth  computing 
x(t  +  ijt)  and  ll't)  in  an  eaqsllcit  fora. 

%  ahall  coagiute  x(t  +  l|t)  by  inlvtotlon,  auppoalng  that  x(t{t  -  l) 
ia  known.  Bia  cosaditlonal  ffsgwctatltv  of  (l^)  with  rsapcct  to 
l(^q)s  ^  deccagioaed  into  two  parts; 

(l)  the  condition  eotpectation  glvdn:  s(t  ),  ,,,,  z(t  -  i;,  «urtd 
(2}  the  ccaidltlaQal  expectation  glTen 

^(tjt  -  1)  -  a(t)  -  H(t)x(tjt  -  l)  =*  -  1)  +  l(t).  (11.5) 

gausBlan 

Bkeaa  two  aeta  of  ireadon  Tarlebleis  ai**  Independent  j  hence  tbe  candltlcne.1  r-xpee 
.  tatlona  aBy  be  consnitad  8epf.«itely  (see  (B.  I’l)).  Taking  coitditirnr^i  cxpectH- 
tiona  OR  both  aides  of  (X^)  with  reapoct  to  s(fc),  x(t  »  l),  ...  yields; 


4(t  +  l|t)  -  i(t  +  1,  t)£(t|t  -  1)  +  C(t  +  1,  t)Y(t{t  -  1) 
+  »(x(t  +  l)ji(t|t  -  1)}, 


i 11.^^ 


W*  have  aeae  alxwaly  that  ^tjt  -  l)  «  0. 

yh  eoaewte  the  casdltlonal  cxpeetatlcm  Ir.  (ll.  k)  with  the  aid  of 
Btooitai  (B.6).  For  thla  purpoae,  we  need  two  covarionce  wtrlct  s.  Toe  firMi 
of  thaae  is 


COT[x(t|t  --  .1}]  =  COT[g(t)J(t  )t  -  1)  -f-  ^(t)]j 


siiaccr  3r(t)  and.  x(t)  are  jj>iopendent. 


Bjc  o«ser  mtrix  ii,  hf  (I^), 

covLxCt  +  1),  »(tjt  -  1)3  -  cov[f(t  +  1,  t)35;(t)  +  r''t  +  x,  t)ir(t),  -  l)j, 

-  co»r»\t  I-  1,  t;x\i;jc  -  x;,  -  Jl)], 

+  corCCCt  -  1,  t)v{t),  r(t)3,  (11.6) 

»  +  1,  t)E(t|t  -  l)H’(t)  +  r(t  +  1.  t)C(t). 

ECx(^  l)li(t|t  -  1)3  -  J!(5(t  +  ijt  -  1),  -  1))  (ii.7) 

»  t«(t  +  1,  t)E(t|t  -  l)H’(t)  +  r(t  +  1,  t)C(t)3[H(t)E(tlt  -  l)H'(t)  +  R'(t)3‘^i(t|t~l), 

CoBiblaias  (11.5-7)#  w  obtain  the  equatloroe  odP  the  fllt-eri 

i(t  +  xlt)  -  »(t  +  1,  t)4(t|t  -  ll  +  5(t)»<t), 

ubere  ( 

} 

K(t)  o  [♦(t  +  1,  t)E(tjt  -  l)H'(t)  +  r(t  -f  1,  t)c(t)][H{t)5(tjt-l)H»(t)  +  5(t)3;|(ll^) 

f(t  +  1,  t)  -  t(t  +  1,  t)  -  l£(t)H(t).  I 

or  ooxurae,  the  Initial  state  -  l)  ot  (H^)  jauat  spoclflfid 

aJjaOs  ads  1«  to  be  taSswi  as  aero,  since  Initially  thers  ere  no  obsermtioaa 
and  the  mean  ot  x(t  )  Is  zero, 

‘ftwi  general.  bloOt  Clagyaa  of  the  filter  lo  shovn  In  ?ig.  '3.  It  ie  a 
ffiedfcacJt  systetu  built  aromd.  the  taodel  of  tJj?  randcro,  i^'-rnitiace  (Ij,K  ’^'S  error 
algMG.  z(tjt  -  l)  la  fed  for^sard  Into  the  aodel  vlth  ^ia  «(t).  HS.'ie'.  galr. 
is  such  that  the  input  to  the  nodcl  is  the  conditional  ea^iec  text  ice  at 
x(t  l)  glren  the  obserred.  dlJTerence  z(t)  -  s(tjt  -  l).  One  part  of  this 
condltioml  expectation  is  duo  to  estloatlng  .x(t  +  xj  t  -  1),  and  the  other 
part  is  due  to  estimting  v(t). 
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Bia  values  of  f(t-  +  l{t)  is  kacm  iawedistai^  after  tisaes  t,  htct  it 
la  not  needed  for  oooqpuilng  the  next  eatlmte  lutil  time  t  l.  ^la  tijce 
delB^sr  aaOna  It  poaalble  to  psrfora  the  c«aQ?vtt«tjlona  indloeted  hy  (u.^). 

!2!he  aaignltude  of  K(t)  la  Indlentlve  of  the  oaKnmt  of  infom&tion 
cootained  In  the  al*S£l  2{t|t  -  1)  iteut  tJSe  rtete  x(t  -4-  l),  Kila  fsropertjr 
of  X(t)  oan  he  uodd  preolee  hecauee  the  queeititj  cf  Infor^tio;:;  Isi 
of  Shannon  can  he  eigpllcltly  oaloulated  for  iauaalan  randca  proceseea  [s^]. 

One  can  then  ehov  [26]  that  X(t)  ia  to  he  detemlned  in  auoli  &  my  ae  to 
OMUClBlBC  the  Infozwitian  caareyed  hy  i(tjt  -  l)  about  x(t  -t-  1). 

We  cxxRSilete  the  solutlCKi  of  the  filtering  problani  by  dariTlng  a  mcor- 
alon  relation  for  the  6csadl>^^o^.  covariance  natrlx  vhic^.  la 

the  only  remijulng  ucdmovn  in  (H^).  Bile  can  h*  obtalsted  hy  injsp?*^^loes  fios* 
Steorea  (B<  1^),  rewBnbering  that 

cor[x{t  *  ijt  -  1)]  -  cov[f,(t  +  1,  t)x(tjt  -  l)  +  C(t  -f  1,  t)e;{t)j, 

-  i(t  +  1,  t)§{tjt  -  l)i*(t  t  3,  t) 

■*■  r(t  +  1,  t)ft(t)|:»(t  4  1,  t)* 


mu,  by  (B.13) 

£(t^i  |t )  -  i(t.+i,t)  {Z(t  |t-i)  -  [£(t  |t-i)H*  (t )-tr(i>fi,t)g(t)  ]  [x(t)s(t  I  Ct^»s{t)  j  • 

X  [i(t)£(t!t  -  1)  +  c'(t)r*(t4-i,t^)«;Ct~i,t) 

+  C(t-a,t)t(t)5^[tfi,t), 

We  ahall  call  (HI^  )  the  nartgice  e^tuptlgw. 

Several  xeatares  of  this  s^rsytlsa  we  notcvmacthy. 

Tirmt.  the  ei;^tloa  doea  not  invcOLve  the  ohaervatlaeio  SfeLa  ie  a 

apeclal  ppaperty  of  the  mltlvorlate  putaaieun  earJbttfesatfsJ 

covariance  'aatrix  dooa  not  «3*pen!d  on  xissi  valuaa  of  fLr«  cowlttlSiaslsss 
Slnr*?  the  galna  of  the  oiptlaal  filter  are  ■ovezned  ty  the  wriasKw  «Jgsfeti«y 
thi*  awfitaia  that  the  atructKrn  cf  the  oftlwl  filter  Ite  elaagsat 

con  bo  datemlnrd  Indepsndeotly  of  the  .xvadoa  data  ^.(t). 

Second,  equatlona  (H^  -  -  1?^)  tcgarther  rarvlartely  islwcBlao  tses 

conditional  dlstrllniiicn  of  the  zandcn  se^iitance  for  all  t  h  t,  glvesu 


{m-) 


•••p  ^  othftr  vox^dSi  tbd  quuitltlfts  x(t|t  -  1)  auad  ^{t|t  -  X) 

«ipp<Murl»g  In  (ll^  -  )  My  regwrdnd  as  th®  rtat®  of  the  f ilterJjig 

problem:  the  oondltion&l  dlstrlbatiaoB  eaa  be  epocli'ied  by  a  finite  n\aaber  of 
p&rsMtsrs.  Shic  happy  state  of  affairs  is  due  to  the  gauaslsn  and  aarhovlan 
assuBQitionB.  Siere  are  no  other  cases  hsioim  at  present  "wii&m  the  aotiic  Itionol 
diatrlbutlons  con  bo  specified  vith  ccecarable  ali^llcity*  this  is  preoiaely 
vhore  the  basic  difficulties  of  the  nonlinear  prediction  and  flltorlnt;  problems 
lie. 

5hlrd,  the  ver5.«i,‘ oe  equation  4»  ,1uet  another  form  «f  the  ccLabrul-'d 
Wiener-Hopf  equation  [1-5 ]•  (S®e  [5]  for  «•  detailed  dlscuBSicn  of  tho  vector 
form  of  tho  Vlenor-Hopf  equation  in  the  cbntinuoxMs  case,)  This  equatron  states 
that  x(tjt  -  1)  and  x(t|t  -  l)  uncorrelated  (orthogonal)  random  varlablesj 
in  Other  vordo,  the  variances  of  x  and  x  add.  Bie  variance  equation  is 
Just  one  of  mary  ways  of  expxessing  the  muss  thing.  variance  equaticEi 

for  random  pi'ocesBes  can  be  derived  directly  from  the  Wienor-Hopf  equation 
as  W518  done  l.n  (5],  The  variance  equation  is  also  closely  related  to  the 
calculus  of  variations,  as  vlll  be  -llscvuisod  further  in  Sect.  15. 

ro\irth,  the  solution  of  the  variance  equation  is  not  determined  until 
the  initial  sta+e  E(t^|t^  -  l)  is  given.  This  should  be  T—sajarded  as  part 
of  the  problem  stateiMnt,  since  obviously  £(t  |t^  -  l)  ••  E(t^)  »  cov[x(t  )]. 

To  avoid  any  possible  misunderstandiTig,  let  ut  mention  how  S(t^]t  -  l)  Is 
determined  in  th«  coventional  Wiener  theory.  There  it  is  assuned  that  the 
random  sequence  is  ctatlonary,  in  othSr  words,  w(t  +  I,  t),  r(t  +1,  t), 

C(t)  Eue  constents;  moreover  ♦  is  a  stable  matrix. 

QSien 

t-1 

x(t)  «  r.  lit,  f  +  i)r(T  4  1,  a)w(t)  (11.8) 

X“-oo 

lo  a  vell-deflned  raniom  vector  with  zero  msan  whose  covariance  matrix  S  is 
lri4-»>endent  of  t  and  can  be  readily  calculated.  Thus  E(t  .^lt^  -  l)  =  S; 
while  not  explicitly  given,  the  value  of  £(t^  |t  -  l)  is  implied  by  the  usr-ustr. 
tions  of  tho  problem.  Pinally,  If  E(t^)  is  nonnegative  definite,  then 
E(t  +  ij  t)  is  also  ncamegative  ilefinite  for  all  t  h  This  lo  obvlouo 

since  E(t  -f  ijt)  lo  a  covariance  matrix. 
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(n.9)  zoumm  oi  ihe  fiiothiir>  prohikm  for  rahixm  skquitoes. 

Under  the  asBunoatlonB  of  Sect.  9,  the  aolutlon  conaiata  of  calcalLatXng  tlse 
conditional  ftamaetftticne  and  ccnditlop*!  ccrv»rlw:iCdB.  tyy  a&aaa  of  gqmtiogis 
(234  -  IHj  -  174). 

•Sxc  eofiditloiml  »sa«aB  are  eoaputod  tor  the  ’optiJBBJL  filter’  (H^)  idticfa 
Is  a  faadbftok  eygtem  vita  Its  Immi  “being  tbe  observations  a(t),  gie  initial 
atate  of  tto  filter  1b  ^(t^  |t^  -  l)  ■  0. 

Bbte  coDdltlona^l  varlaaceB  are  golutloGB  of  th*  TS.rlaaee  sguatlon  (lU^) 
and  are  calculated  independently  of  the  obeervatlong  s.C'*')*  1!be  ocaadlticml 
yar:^aaco6  j^teralpe  ttee  mis.  K(t)  of  tha  optlaa.!  filter^  'Sus  Initial  gtate 
E(t^)  of  variance  c^isaatlcn  la  glyan  ag  nart  of  the  protlaa  etg.tgxaaat, 

;pie  solution  of  tha  flltarlrut  problfwa  ia  given  In  a  canvenlent  fora  cgily 
if  t^  a  tj  then  x(t  +  l|t)  and  S(t  ijt)  contain  all  necoHsary  iiiforaa.- 
tion  for  conputlnff  tl>e  eopdltlonol  parobahility  dlatributiopa  of  the  futm^e  of 
the  random  eequance .  x(t). 

able  reoiilt  vas  first  obtained  hy  l&QjBan  [4]  In  19^  except  for  a 
slightly  less  general  problem  statenent  and  the  unneceacary  acsumptioji  that 
the  Inverse  of  the  covariance  matrix  of  i(t|t  -  l)  exists.  The  latter  diffi¬ 
culty  Is  nov  eliminated  by  the  use  of  the  pseudo -inverse. 

In  the  ccssveatlaoal  Wleocier  problem  se  assume  that  ^  ^  ^  ^  C 

are  constemts;  In  addition,  t  1*  token  as  -  00,  In  this  case  the  variance 
equation  (XII^)  should  he;ve  a  constant  nomegative  definite  sol’Jtiosi  (equili¬ 
brium  state)  ^  to  atilch  vlU  correspond  a  constant  gain  ^  and  therefore 
a  constant  optimal  filter.  In  Sect,  I6  -m  sliall  discuss  the  conditions  under 
which  E  exists,  Is  \mlq[ua,and  is  the  limit  of  erery  solntlon  of  the  narionoe 
equation  which  starts  at  a  ncaujegattvio  definite  initial  state,  ¥e 

hasten  to  point  out  already  here  that  this  is  always  the  caae  If  m  add  the 
last  remaining  assumption  of  the  Vlanex’  theorvt  the  model  is  asymptotically 
stable.  Hence  mder  the  ctxiventlanal  asstBaptlcns,  the  solution  of  the  Wiener 
problem  reduces  to  the  dsteralnation  of  the  unl(iue  equllibrluai  state  E  of 
(m^)  which  Is  purely  an  algebraic  probleia  Involvlnit  the  solution  of  slmnl- 
tanec' :  ^  .xd,.-itlc  aquations,  ms  cou  ue  oarrlev^  ouv  explicitly  only  in  simple 
cases  and  will  be  discussed  extensively  In  Sect.  12. 

Bxe  chief  reanlnlng  task  in  filtering  theory  is  tha  studly  of  the  varl- 
anc<t  equation.  Sils  Is  dltfloult  because  the  equation  Is  nonllnoar.  Hxe  pro- 
bljsm  can  be  best  appreciated  from  the  study  cf  detailsd  cuns^leM.  ’fwo  of  thase 
are  given  in  Sect.  12.  A  sumaary  of  idtat  Is  known  ahout  tilrw  queLUtatlve  behavior 
of  the  variance  equation  appears  In  fleets.  l^-l6. 


ip£ 


12.  of  dliegrU  flltMlag.  In  tlie  tvo  «xjnQ»liB&  dltotui»«d 

h«rtt  Vs  assia*  fat  aiaipllclty  that  0(t)  is  Identioally  sera.  IMc  will  not 
entail  a  great  loss  of  generality.  Ve  ahall  vrite  S(t)  and  instead 

of  5(t|t  -  1)  ax^  ctj,(t|t  -  l)  to  save  space, 
file  stapleet  possible  case  is  the  foUoving) 

(12.1)  BCAMPIX.  Conalder  a  constant,  first  =orisr  swdsl.  Satting  the 
constants  and  equal  to  1,  ve  have; 


file  variance  eqiiation  follows  by  Inapection  from  (HI,): 


P  o?i(t) 

+  1)  -  <Piif®ii<^)  “ 


ai3_(t)  4 


]  +  q-,.  (12.5) 


file  equation  of  the  optimal  filter  it; 


+  l(t)  .  -  1)  +  - -  !2,(t)  -  Xj(t|t  ')11  (12.V) 

dyCt)  +  r 

©lere  are  several  cases  of  interest,  depending  on  the  valv^^s  of  the 
parameters  q^,  and  r^. 

Case  (l);  ■  0.  Squatior.  (12.5)  ■’aroediately  reduces  to 

<j,  ,(t)  K  q  »  const,  for  t  >t  ,  fiierefore 
11  11  o 

+  lit)  -  »j(t). 

In  other  words,  the  filter  has  no  rcciaory  and  the  bent  estlnuite  in  the  last 
piece  of  data. 

In  all.  other  cases,  the  transient  behavior  of  , (t)  will  be  wore  com¬ 
plicated.  To  nnalyz.e  it,  let  stand  for  on  cquilibriiim  point  ox  syatem 

(12.5),  defined  by 


(12*5) 


(  ^11  11 


■)  5^  + 


'11 


‘11 


Of  couTM  the  requlreiaent  k  0  tnuat  he  satisfied  aluo. 
Ve  define  deviations  from, equilibrium  by 


^11^^^  “  ‘  ^11* 


S\ibstltu:..jr.ii  this  into  (12.5)  and  UBin«  (l2.5)  give  a 

r  r 

BOii(t  +  l)  *  ^ - ^  f  ‘ 


11 


‘'ll  ” 


^11^^^  ^11 


(12.6) 


We  ore  now  ready  to  discuss  «ie  remaining  cases. 

Caee  (ll)i  >0,  "  0,  i  1*  Equation  (.12.5)  has  only  one 

solution,  which  is  5^^^^  •  0.  If  o^(t)  ■  50j^^(t)  >  0,  theii  the  fac^'r  on 
the  rl^t-hand  side  of  (12.6)  Is  always  positive  and  less  than  1.  Hence 
5q^  T (t)  decreases  monotonlcally,  and  ell  solutions  of  (12.3)  converge  to  0 
if  they  start  at  0^(1^)  k  0.  Negative  valuer  -.f  JTii(to)  are  of  course 
ruled  out. 

Case  (111);  r^^  >  0,  -  0,  >  1*  New  (12.5)  has  two  solutlonc 

5^  =  0  and  -  (9^^  -  l)rj2*  Substitute  the  second  value  cf  into 

(12.6;*  then  the  factor  on  the  right-hand  side  of  (12.6)  is  leas  tliaii  one. 

Blue  |c^(t)  +  (9^  -  decreases  ocmotonically  and  every  solution  of 

(12.5)  with  >  0  converges  to  the  second  equilibrium  noln-*.,  ’Use  wily 

exception  is  the  oolution  which  Jo  an  unatuhle  equili¬ 

brium  point. 

Case  (iv)t  r^  >0,  >  0.  Now  equation  (12.5)  ®  single  solution 

5  .  Ihe  first  factor  of  (12.6)  Is  less  than  one  ac  a  consequence  of  (12.5)* 

The  seconiJ  factor  is  less  than  or  ^oual  to  1-  lfi<jjj^(t)  |  decreases  monotonl- 
calJy  and  all  solutions  converge  to  the  unique  equllibriinn  point  0^  ^ . 

What  can  be  said  about  the  stability  of  the  optimal  f  13. ter?  In  C&se  (i  ), 
this  question  is  vacuous.  Otherwise  the  1x3.  transitJ.on  matrix  oi  the 
optimal  filter  Isl 


I5I 


♦ll(t  +  1,  t)  - 

In  Case  (ii),  tends  to  (p^j^  as  t  -*  oo;  in  Case  (lii)  ,  tends  to 

In  both  easea^  the  optimal  filter  is  asymptotically  stable  'onlesa 
j(Pll|  a  1.  In  Case  (!▼),  the  optimal  filter  ia  always  asymptotically  stable, 

V*»  ^  1  ^  ^ 

\  aitfi  9  y  / 

lim  UnCt  +  1,  t)|,  « 
t  -♦  oo  '  ■ 

(12.7)  EXAMPLE.  When  the  model  (12.2)  of  the  random  sequence  Is  non- 
constant^  the  discussion  is  similar  but  much  less  elementary,  llie  inain  point 
is  this:  we  must  assume  that  the  parameters  ^  '^)l> 

v^^(t)  describing  the  model  are  roughly  of  the  same  order  of  (magnitude  at 
all  instants  of  time;  in  other  words,  they  cannot  become  arbitrarily  large 
or  arbitrarily  small.  A  convenient  condition  assuring  this  is: 


^ii‘n 


'’ll  ’’ll 


<  1. 


0<«1  ‘ 

l<p^(t+l,t)l 

S  ^^  <  00, 

0  <  Og  S 

S  <  oo, 

(12.8) 

0  <  Oj  s 

r^(t) 

S  p.,,  <  oo. 

5 

We  shall  assume  (12.8)  for  the  sequel.  What  hapi)enB  when  these  conditions 
are  not  met  remains  aa  opct»  problem. 

An  immediate  conseque.  ue  of  {-21.3)  is  that,  even  thougli  ‘^11  ^  ^ 

is  arbitrary, 


2 


S  a  f-*- ' 


■  ^"2 


for  ail 


n\ 


In  other  words,  the  solutions  of  the  variance  equation  aye  unifomly  bounded. 
Using  the  variance  equation,  one  gets  Immediately  the  Inequality 
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evtry  solution  of  the  varlioice  equation  till  tend  toward  sose  parbiculAr  soXu^ 
tlon  5jj^(t)  coatatned  in  the  ttiglon  (12«9)*  BMs  aolufclon  is  conveniently 
dsfined  "by  toeing  ite^  s1^rtl|^|^lnt  ■  0  and  t^en  letting  t^  -h> 

®>e  function  5^(t)  ^  neaauJdad  ns  the  "moving"  equllihriua  state  of 

the  variance  equation. 

We  carried  out  the  dlscuBslon  In  bo  much  detail  In  order  to  indicate 
the  method  of  proof  in  the  general  case.  Even  thou^  the  varlaiice  equation 
iB  nonlinear,  its  transient  hahavior  can  be  studitsd  cciiVun  iently  by  ae^e  of 
fonsuln  (12JJ0)  and  itss  garierallaatlons.  3ae  Sect.  16. 

Qie  next  exauplo  concerns  a  second-order  model;  this  sli^t  liicrease 
of  ccasqplesity  makes  the  explicit  discussion  quite  involved,  even  for  the 
steady*4itate  bebuvior.  ' 

(12.11}  EXAMFUS.  Consider  the  random  seqrjence  x(t)  generated  in  the 
following  fashion; 


x(t)  -  k(t)  +  m(t), 

where 

k(t  +  l)  «■  k.(t)  +  w^(t), 
ta(‘,  +  l)  n  iii(t)  +  n(t), 
n(t  +  1)  -  n(t)  +  w^Ct); 

w^C't),  WgCt)  are  gauaslan  white -noise  sequences  with  zero  raean.  In  other 

words,  x(t)  is  the  am  of  two  random  sequences:  one  witii  independent 

gaussian  random  increments  (first  differences),  and  one  with  Independent 

gaussian  random  second  diffeipenceo.  Moreover,  values  of  x(t)  are  raepowred 

with  an  error  •^(t)  which  Is  also  a  gaussian  vblte-nolBc  8equ3nce  with  zero 
* 

mean.  Uiius 

z^t)  «  x(t)  +  v^(t) 

It  is  easy  to  see  that  =  x  and  Xg  =  n  is  a  sultabxe  definition  of  the 
state  variables  In  this  case.  Ihe  matrlcb?  in  (  l^)  at'e; 


#  - 

'The  llTBdt  iim  o,  ,(t:  0,  t  )  oLwavs  oxistBl  see  Sect.  lb. 

j  11  o 

t  »  ~  oo 


t 


1 

1 


1  0 
0  1 


B.[l  o] 


variance  equations  eaeil 
0„(t  +  l)  ■  ^ 


r«  ./-t-A  4-  rt 

~  •  ''1C?'  - 


'12' 


"■  "'ll 


a. - . 

■X.1' 


fl^(t  +  l)  «  ■*■  ~ 


c^_(t)  4  1. 


11 


[e^(t)  r 


cTggC*  +  ^)  “ 


The  optimal  filter  is  given  by 


x^(t  4  l|t)  =  X3^(tjt  -  1)  + 


XgCt  f  ijt)  =  XgCtlt  -  1)  4 


0,-(t)  4  0f,p(t) 

1)  4  X,(t|t  -  1)  “ - 

"■  ’^ll 

-  [z,(t)  -  Xi(^K  -  1)  ^ 


x^(tlt  -  3/] 


^12^^^ 


11 


Die  detailed  amlysiP  of  thia  exanrp.le  1b  ao  tedltme  thax  vc  shai.L  ccnaldc 
only  the  steady-ntato  behavior.  In  other  vordo,  ve  aball  aimly/.e  the  v^qui Li¬ 
brium  BtateB  of  the  variance  eqxjntiona,  ijlven  hy 


-  2 

^2^12  ^  ^22  ■"  "u^^^ii  ' 

(  ^  ’>  1  O  1 

’  id.-  ^  "12^^11  ^1?^-’ 

i\2.1j) 

(12. Ik) 

and  Bub.lect  to  the  condition  that  the  oteady-otatc  variance  raatrijc 
be  nonnegattve  definite,  which  will  be  tinie  11  and  only  If 

1^ 


^11  * 

(12.15) 

(12.16) 

To  avoid  discussing  euBibBrBOBS6  special  cases,  we  aesuiEc  that 

-,  &re  &ix  pvoxoxv^s*  xn^jL^^uviAdL^  ow  uluOx^^v jcsuxwnu 

n 

^  1  * 

a  »  q,  /r„  >  0,  0  ^  >  0; 

eliminating  i  and  relations  (l2.il,  3^.15,  1^12/  12.15,  12.1b)  ixjcrav--' 

respectively 

0 

t  “  n  -  1, 

(12.17) 

rj5  =*  •/p(t)^  -It  T)) 

(12. IB) 

7^^  ^  '/pT^  -  (2  +  a)Ti^  -  \^TJ  +  1  ■  0, 

(12.19) 

1  »  0, 

(12.20) 

tC  »  PTJ^ 

(12.21) 

By  (12.17)  an'l  {l2.20)  i  1.  'this,  (12.21),  and  p  >0 

J  >0  and  5  >  0.  Cooiblnisg  (ijd.lf)  oii&  (12. iB),  (12.21.)  b«e<j«ea 

iTaj^ly  that 

nCT  -  \  f' 

(12.22) 

With  (12.17) >  this  yields 

T)  >  Ic 

(12.23) 

Turning  now  to  (12.19)/  notlo®  th*  0ymu>try  of  the  coptTtcisnt#. 
B\iB  T3ES?ms  tfet-  if  T}  !«  a  root-  then  l/fj  is  also  s.  root.  fe>ro  is  !s«v’sr 
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root.  Denoting  ty  lAv  rootp  of  (12.19)  . 

obtain  the  following  conditions t 


(\  +  V\)  +  +  V^)  “ 

(tJi  +  V\)  *  (\  *  “  ~  ^ 


(12. 2U.) 


ISilE  IB  aqulvalant  to  a  qiwidraitic  oq»ietlon  .tn  the  uifai'^ovi-  i;^  ;  1/ 
vhlch  has  tho  DolntlonJ 


In  view  of  (12,23)  ve  auat  choo&a  the  +  algn;  the  ni.n-.iB  eigii  vili 
tt“u  corroapond  to  +  l/T]g  in  (l2.2l^).  Oolvlng  (12.25)  for  w  got 

Tu  -  ^(  -/p  +  -/iS  +  Itot  +  p  +  +  »4a  +  2  •/l6  +  J4a  +  p)p.) 

•Kie  root  conroaponding  to  the  -  al@:j  le  the  reciprocal  of  the  root  correo •• 
ponding  to  the  +  sign.  In  view  of  (12.23)  we  muat  choose  the  larger  r<x)t, 
80  that 

Tj  =  «/p  +7i6T~ii^  p  +  -Zap  +  %  +  2-y(i6  +~5Qr+”pip).  (12.26) 


iB  tne  only  root  of  (l2.19)  which  co’xld  lead  to  a  po*>itivc  definite  ciatrlx. 

It  rcmainB  now  only  to  check  whether  (l2,22)  holds.  By  ^.12. 26),  i.fid 
(12.^)  WB  ba'«e 

(’J  “  »  i  (20  t  p  +  V  Cl6  +  ki  +  p)p)h2/p  >  VpvV 

H  *■ 

Hence  m  here  prfrvwA  tlgure  exHata  cmc  and  only  one  solution  of  (12.19) 

^jeh  le  nonneaetlve  deflcltei  thle  eolutlan  is  actually  positive  definlr-e. 
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(12.17»  12*l8,  12.26).  It  caa  bo  shown  (see  Sect.  16}  that 


^vSlaiS* 


ua  »t»ts 


^  ^3£'  °22- 

Althouj^  this  probiea  appears  to  be  qnite  aleaentary^  the  author  is 
not  aware  of  any  detailed  study  of  it  in  the  lltemtuj>e.  As  a  setter  of  fact- 
in  a  recent  note  to  the  Soviet  Acadenv.  A.  L.  Bnmdno  and  a,  t..  f;in+.«  TotI 
errone^w^  assert  (without  proof)  that  the  Bolutiaa  of  thj.s  problem  ia  not 


■LOi 


13.  gf  thit  jfllt«,glixg  pwblaa  wgidOM.  proceaseB.  The  raaln 

Ot>j6et  of  tM.9  MCtioa  is  to  ostfilallsh  ralaticaa  anailosous  to  (ll^)  euid 
(m^).  A  ngorou*  proof  of  this  sRist  he  preeadod  "by  a  rtgoroiw  definiticii 
of  the  yhl^-aoiso  process®#  la  (I  )*  not  do  thie  here  hut  will 

fm  .*  '  C 

&ppct&l  to  the  #«nai-ri50roniB  llitltlng  argieaents  elreAdy  used  in  Sect.  6.  A 
different  derivetloa  (rlgoirotts  except  for  the  \i9o  of  delta  functionB  in  the 
definlttoa  of  the  covarienee  Twtrlcee  of  white -nolee  processeB)  nsky  he  found 
Hi  [5]. 


As  In  Sect.  6,  let  q  be  a  pocitivo 
Aificrste  so  tbat  its  successive  values  ditffer  hy 


JL  j  ^  ^  V 

MJLUA  W  UiKt  Iw 

!}  .  auuumixif^ 


♦  ie  the  transition  watrlx  of  a  continuous •’tiise  linear  dynamical  syntera 
and'  r  1*  given  by  (4,10),  we  have 


#(t  +  q'^,  t)  -  I  -«•  q"%t)  +  o(q''^)^ 
r(t  +  t)  «  +  o{q‘^)/ 


In  view  of  the  discusHlon  of  Sect.  6,  the  covariance  mtriceo 
and  R(t)  In  (III^)  ^>re  to  he  replaced  by 

q’^c(t),  q"^(t),  and  q'^nCt) 


AS  q  ->  00, 


Substituting  these  expressions  in  (111^^),  ve  obta^uj 

^  ~  . I  m  j(t)2:(tjt  -  q~^)  +  s(tjt  -  q"^)|:'(t) 

"f" 

[E(tlt-q‘^)H’(t)  +  g:(t)C(t)][q"\(t)£(tjt-4'^)H»(t)  ^  H(t)  ]  [H(t)£(t  '  (t.)G’ 

+  a(t)a(t)Q»(t)  +  o(q“'^). 

■f*  ••1  t  t 

Since  foA)  «  a  A  IJT  Of  /  0  but  0^  «  0,  ve  miet  be  careful  not  to  In¬ 
troduce  a  diaoontlnuity  in  the  term  [  vfailfi  taking  the  limit  q  oo. 

f 

Ihe  trouble  is  most  easily  avoided  by  asounilng,  cnee  and  for  all,  that 

R(t)  Is  p«>eltlve  definite  for  all  t.  (I'^i-l) 

Btaslng  to  the  lijmit  q  -  oo,  no  obtain  the  verisjice  evjuetlon: 
d^dt  =■  F(t)£  +  ZP’(t)  -  [rE’(t)  +  0(t)C(t)]R“^(t)rH(t)£  +  G'(t)Q’(t)] 

■*.  a(t)Q(t)0’(t).  (ITI  ) 

trtiose  solution  is  the  covariance  matrix  E(t|t). 

The  sane  limiting  process  applied  to  (H^)  yields  ths  equations  of  the 
optical  filter  in  continuoua  ti«s: 

d^dt  -  F(t)x  >  l(t)[T(t.>  .  x}j 

(  (II.) 

5(t)  -  ;E(tl‘)H'(‘)  2(^ )?(*') ;a‘^(«)-  3 

The  solution  of  the  n^ve  differentiAl  equation  la  iciic  ccndltiiifi&l  expectation 
x(tlt). 

We  have  already  noted  in  Sect.  8  that  if  1  alngular  (i.e.,  ec=ae 

llaeax  corialnatlon  of  couq;crN>nts  of  j;(t)  can  be  obsarvud  eaactly)  thawii  the 

l£5 


optimal  filter  may  be  an  unbounded  operator  —  such  as  differentiation  -- 
'Bhich  cannot  be  realload  by  Tssens  of  e  linear  (Synaadcal  systeia.  Hence  condi¬ 
tion  (15.1)  cannot  be  readily  relaxed,  as  is  clear  from  the  expression  for 
the  optljB&l  fft-in, 

^Che  matrix  block  diagren  of  th,e  optimal  filter  is  shevn  in  Fig.  6. 

litemarks  concerning  the  ?^ltlal  conditions  of  -  III^)  apply  without 

modification  to  (ll  - 

c  c 

Equation  (IV gencralizaa  t,ivi:illy  to: 

x(t^|t)  »  f  (tp  t)x(t|t)  for  all  t^  2  t.  (IV^) 

Hence  wb  have: 

(15.2)  SOUITKSl  OF  THE  FIUTEKOUQ  PROBLEM  FOR  RAHDOM  PRDCEOOEG. 
the  aeaumptions  of  Sect,  9  and  the  solution  ccpslats  of  calculating 

the  conditional  expectations  and  conditional  coyarlancec,  by  mejms  of  ocuationa 

<n,  •  ni^  -  IT^). 

The  conditional.  Tarlapces  l(t(t)  ere  solutions  of  tfie  variimee  squa- 
tlcn  (m^)  igitd  aire  calculated  independently  of  the  obaervationH  5,(1).  Tite 
rnpd vaxlapoes  determljaa  the  gain  K(t)  of  the  optimal  filter-  ’J5"" 
a-^t.e  E(t^|t^)  Of  the  TariaTiCe  equation  is  given  es  port  of  the 
problem  statenent. 

The  solution  of  the  filteriny  problem  is  givjn  in  convenient  form  J y 
If  d  t;  then  x(t|t)  end  Z(tlt)  contain  all  noceosary  Xr.foruatlofi  for 
-  the  ccaiditionel  probability  digtributiana  of  the  future  of  the  random? 

process  x(t). 

This  result  was  first  published  3n  [5]. 

Although  Bieoram  (13.2)  appears  to  1*^  ccmpXetf^ly  analogous  bo  llicore© 
(11.9),  there  is  one  major  difference:  the  'solution’  of  the  problo®  in 
TbeoroK  (I3.2)  Is  tied  to  obtaining  a  solution  of  the  varlbnco  equation  (ITX^). 


.  (iXIg)  sattanaa  r  Upaehlta  cond.lt ion,  It  foUova 

that  aolvttlona  of  (lll^)  viH  exlat  for  arbitrary  ?(t^{t^)  In  soaa  amll 
lntegva.1,  tine  contn^^nln^  t^.  But  it  la  not  claar  vilAout  furtoar  tnvaa- 
tlgation  th&t  solutions  exist  for  all  t  *  (As  a  asttar  of  fact,  this 

way  not  even  be  true  for  arbitraiy  2(t^jt^),)  However,  we  can  re*dl.ly  show: 

(13.5)  U  £(*^0^0^  noy.<?gative  definite,  then  has  a  ias,X<nae 

Bolutlon  which  exists  for  a3-l  t  ^ 

This  aay  be  proved  as  follows.  .  let  r.(t)  be  the  covai’lance  natrix  of 
x(t)  defined  by  (lg)j  if  the  covariance  matrix  of  the  initial  state  ^Ct^) 
is  ?:(t^)  =  5(tQ|tQ).  Utilising  the  formulas  of  Sect.  4  and  rectdlinir  that 
2(x(t)}  =0,  we  have 

E(t)  -  cov[x(t) ]  , 

-  «v%\,)E(t^)®’(t,t^) 

t  t 

+  KC  /  dT  /  dT»  ♦(t,T)G(T)w(T)w»(T')G'(T')®'(t,T’)] , 

to  tq  “  “  " 

t 

=  i(t,t^)i;(t^)i‘(t,t^)  +  i(t,T)g(T)3C'r)c'(T)$.'Ct,t)dT,  (l3u4) 

!Ihl8  vhovs  that  2(t)  Is  bounded  ^dianever  t  3:  t^.  But  tlie  definition'  of 
conditional  covariance  matrix  (see  B.lj)  shows  that 

E(t|t)  «  E(t)  fox-  all  ^  31  t  •  (15.^) 

in  other  words,  it  is  Itnown  a  priori  that  aoluttons  of  tl»e  rsa'i-ance  equativni 
must  be  bounded  for  any  t  a  t^.  Substituting  this  faci,  into  the  standard 
cxisten-^e  proof  of  solutions  of  nonlinear  differential  '-aquations  [Ihi  provea 
(15»5)*  It  should  be  noted  that  the  argument  leallng  to  the  Inequality  (15. 5) 
does  not  hold  if  E  is  not  a  covariance  matrix,  in  other  words,  if  2(t  )  ic 
indefinite. 
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At  first  It  aay  appear  that  the  aolution  of  the  nonlinear 


differential  equation  (IHq)  vould  in  general  require  nuacrlcai  quadrature  —  a 
dlsagreeabiB  proepact  beoaiiso  of  the  n(n  +  l)/2  -mrlaoieB  involved  (which  are 
the  distinct  elements  of  the  n  K  n  sTfflssetric  tsatrix  S).  !3ut  is  not 

a  general  nonlinear  differential  eqiationj  it  is  a  very  special  one,  the 
natrix  riocati  equation,  which  is  well-known  from  the  calculus  of  variations. 

Ue  shall  utiliise  this  fact  to  derive  an  exact  formrila  for  the  soluti-one  of 

Consider  the  bamlltonlan  function  defined  by 


2i/  (x.2,t)  -  -ijg'(t)x||^^^  -  2p»F'(t)^  4  :!H(t)C’(t)G‘(t)xf  (1?.6) 


f 

and  the  associated  canonlcoj,  differential  equations  of  Hsunllton: 


(t)^dt  -  =  -F'(t)x  +  H'(t)R“^(t)H(t)g  +  H'(t)R“^(t)C’(t)G'(t)x 


d^^/dt  =•  -  g(t)q(t)G'(t)x  +  F(t)n  -  GCt)c(t)R"^(t)l£(t)p 

-  G(t)C(t)R''^(t)C'(t)G(t):i. 


Let  X(t),  P(t)  denote  the  unique  paSx  of  matrix  solutiono  of  this  oqimtlon 
corresponding  to  the  Initial  conditions 

X(t^)  =  I  and  P(t^)  ^  (I5.T) 

Then  we  have  the  Identity 

p(t)  *  E(tlt.)X(t)  for  «1]  t  *  t  ,  (1^,0) 

-•  —•  yj  ■ 

which  can  be  easily  verified  by  eubatitutlng  (1^.9)  8-^*1  into  (l.:>.  /). 

We  see  tlien  tiiat  X(t)  oatiefico  the  differential  equation 

dX(t)/dt  =  E-P'Ct)  +  H’(t)R"^(t)H(t)E(tjt)  +  H’(t)R“^(t)C'(t)G’(!.)  ]x(t),  (ip.G) 

which  is  deftned  for  aLL  t  *  because  of  (I5.J).  (in  fact,  (i;»,.10}  lo  tho 
ad.lolnt  cl”  the  differential  equation  of  the  optimal  filter.  )  In  view  '  ' 

^  The  con5)cnentH  of  the  vector  ax-e  c) 


(15»9)>  it  follows  that  X(t)  la  the  transition  Bsatrlx  of  (v,J  and 
thus  ^(t)  is  never  singular  for  t  k  t^.  Hence  (15.9)  becomes 

E(tlt)  -  P(t)x"^(t). 

Let  vss  partition  the  transition  matrix  0  of  the  2n  x  2n  linear  Byatam 
(13.7)  into  n  X  n  blocikaj 


§g2{t,to) 


(IJ.U) 


Otan  (15.11)  can  vrlttcn  explicitly  ac 


£(t|t)  -  tgeiCt,t^)  +  §22^'t,t^)E(t^it^)]t^(t,t^)  +  e^(t.t^)2(t^!t^)rl  (13.12) 


the  solutions  of  the  varifw*  *«juatlon  for  t  a  t  ^  caai  bo  expressed 
exactly  in  terms  of  the  transition  matrix  of  the  baaslltoolan  systea  (15.7). 

■Ebe  connection  between  the  senrix  rlccati  equation  and  the  cezvcuical 
equations  of  the  calculus  of  variations  has  been  luiown  fcsr  a  long  time  [^1^ 
but  it  was  relatively  unnoticed  until  recently  [S^].  Uo  tha  best  of  the 
writer's  knowledge j  [5I  was  the  flrat  instance  in  whlufa  the  reljiticm  of  the 
Vlener  problem  to  the  claasioal  cbXeulns  of  varlstloos  was  explicitly  noted. 


l4.  Bxaaplaf  of  oontlawiia  filierljtm.  Die  mmiber  of  cases  vhere  it 
l8  possible  to  obtain  olosed-form  solutions  of  the  filtering  problem  lo 
suzprislngljr  snail.  Ve  present  belov  sane  typical  tuusples  of  these  cases  | 
other  exsafples  of  this  sort  are  discussed  In  [5].  Being  too  ainple,  the 
p,vw,wT»i«*s  to  be  discussed  here  are  of  very  limited  practical  Interest.  But 
they  are  useful  In  conreylng  insight  into  the  behavior  of  the  vaxlance 
equation  and  they  serve  as  a  ueeful  guide  in  obtaining  general  resalts^ 
such  as  those  presented  In  Sects.  1^  and  3.6, 

We  vrite  J(i;  :  £(tlt)  and  5(t)  st  E(tlt)  for  the  oaite  of  Bxmpli- 
clty^  and  assume  that  C(t)  t 

(l4.l)  EIAMPLS.  What  lo  obrioualy*  the  slmpleBt  filtering  problem 
appesurs-  in  Fig.  ^A.  Dila  la  a  constant,  first-oi?der  system  and  vae  treated 
tn  detail  already  by  Wiener  [30].  Aa  my  be  expected  after  I5  years  of 
progress  in  the  field,  tlae  present  treacramt  is  a  good  deal  aiapler  *1*^5 
more  ganersLL.  Die  discussion  Is  very  similar  to  that  of  Exsaiple 

Die  describing  matrices  can  be  read  off  by  Inspection  from  Fig,  TAi 

2  -  tl],  H  -  [1],  and  R  -  [r^3^]. 

We  asaume  of  course  that  r^  >  0.  Dien  the  variance  equation  is: 

’^11/'^^  "  ^M°U.  "  ‘^n/*^ll  ‘^11’ 

The  optimal  filter  Is  alunfn  In  Fig.  T^B,  vhere 

4^1  (t)  - 


&21  “  ’’as  a®e  that  the  ©quillbrius:  i*tAtea  of  the  vwjciauca 
equatiiijn  are  gjmn  by  the  roots  of  quadrat's; 


Ssnae  0^^  ii  •  yseimtxm,  it  osiat  be  nonnesetlTei  thus  m  ooncltite! 

bM  e_i«igw>  •mirnrim  gtete 

0,,  if  q^^>0  or  If  f^  <  0.  Othenrlae  there  me  tje  eriillibrlwi 
0  iBg  ^11^11* 

In  the  oleaeioal  formUatiao  of  the  Wiener  prchlam,  the  seesege 
proeeee  »iet  tie  stationary.  Ciie  require*  <  0.  Moreorer,  one  auieuBee 

of  oouree  alto  that  ^  >  0,  einoe  othenriee  the  variance  of  the  aeseags 
proceaa  vouLi  he  «ero  In  the  steady  state,  tteder  these  adfo^^bliTsti^j  th'^ 
steady-etatQ  gpeii ,  #'f  the  (^iaal  filter  Is  gl-ren  'fey 


®ii  ■  ®iiAu 


♦  inAu- 


(1^,5) 


and  the  eteady-state  optlaal  filter  la  described  by  the  equation 

djg^/dt  -  +  £^E^(t),  (1*^.6) 

vtM-re 

^11  “  ^11  ■  ^11  “  “  "11  '*■  '  (1^*7) 

In  particular,  (l4.6)  shove  that  tbo  optlaal  filter  iw  alvaye  aayapto- 
tlcally  stable.  Tfciese  resulta  turv  veil  intiovn  [1-5  ]. 

In  accordance  vlth  RsiKark  (14,U),  these  fumulas  contlnxie  to  hold  if 
either  f^  SO  or  >0.  If  on  the  other  hand  >  0  and  0, 

then  there  ere  bvo  pooelblc  equilibrium  states  and  It  is  not  obrious  e.t 
first  uhich  of  these  corresponds  to  the  oolutlcn  of  the  filtering  problem 
with  '*'0  “  "  Inspection  of  the  n  rat -order  no»iiinoar  diffss'rartlri:.  cqua- 
tion  (i4,2)  shOTi'a  that  of  the  tvo  posslblt,  equilibrlusi  states  0  is 

always  unstable  and  ■  af^r^j^  la  eil'^'iys  stable  at  t  -+  00.  Ail  solu¬ 
tions  starting  at  conrorge  to  the  second  equilibrlua  state  as 

t  -♦  00.  Bie  optimal  gain  corroapoTAding  to  the  tsecond  eq[uillbrium  state 

is  positive,  and  therefore  the  optlmaLl  filter  ie  asynq>totlcally  stable.  Hence 


1  Ao 


ghft  op^fcjJiid  aqrototlcftlly  Bt>ble,  except  perhaps 

In  the  trivial  mm  '  4ll  -  0. 

Hote  thAt  rtaltlUty  do«i  not  dapend  on  the  oodol-iteelf  iDeing  stable. 
She  opttM^  filter  slMya  profvldeB  feedback  around  the  rsodel  so  as  to  nahe 
the  oloMd-loop  •yitem  stable*. 

In  this  exaaple>  it  is  easy  to  obtain  an  incplicit  solutlcn  of  the 
v&rlanee  equation.  ¥e  consider  the  associated  Usalltonlan  system  (  Y  ): 


+  (l/f-i-j  )pi  t 

lPl/«  -  *  f3_^P3_.  •  , 


(i4.9) 


Vb  assume  that  either  /  0  or  /  0.  The  other  case  is  trlvieJ., 
Then  <  0  and  the  transitioBi  matrix  of  (1*^.9)  ie  151: 


e(t  +  T,t)  - 


cosh  ?,,T  -  - —  slnh  f,,T 

JLL  %  11 

11 


r  f 
11  11 


siiiXi  f^T 


I  ilk, 10) 


'T~  slnh 
^31 


cooh  f, ,T  +  -  ainh 

u.  s  11 

11 


Applying  formula  (15.I2),  ne  find,  for  t  *  0, 


cOBh  )8lnti  f^{t-t^)+  [(l/r^fj^^)8lnh 


If  Ojj^(t^)  ^  »  0,  then  d^(t)  vajilshfla  Identic 

have,  since  <  0, 

lira  cf,^  (t)  =  - - 

t  -4  00 


Othor'vrlBo,  vo 


9 


By  {Ik.'j)  and  (1U.5) 


lia  €f^(t) 
w  "-f  00 


«■  r 


-which  cbedu  vl-th  the  pre-vlous  conclusionB. 

Althotigh  -the  p«scadljig  derv^lojraeata  pjfovlcLe  a  ,*oaipletc  arid,  explicit 
picture,  the  xeaulting  forgulaa  are  qul-fce  co!!5>llcate<i  and  difficult  to 
understand  intuitively.  More  ioifonaatlon  can  be  gained  by  transcKbJng 
the  of  SsajBpXe  (12.T)* 

(iV.U)  TSiUSnM,  We  corsidcr  again  the  ayatem  ahersm  in  Fig.  JA-li, 
but  nw  are  asB-uae^  to  be  functions  of  tlMe.  Analogous ly 

to  (12.8),  -»»  lairpoee  -the  "unlforalty"  conditionnl 


lf^(t)i<  <  00, 

0  <  Og  <  ^  ^2  (1*^-12) 

Applying  theoe  oonditlaos  to  -the  -variance  equation  (lV,2),  ve  conclude  that 
8(^>0  If  0  s  a  <a^  -  -  t  -t  a^, 

<  0  if  ff  > 


Bence  e-rery  solution  ocf  the  variance  equation  Btartlng  at  a.  (t  )  ii  0  aus' 
eventuttl.'’y  enter  the  regioa 


0  <  aj,  -  «  «  1?  +  c  <  00-  (1^.15) 

provided  e  >  0  1b  suitably  ojuall.  fenoe  it  will  suffice  to  restr'lct 
attention  to  BolutianA  of  Use  -variaii-se  equation  lying  entirely  in  ti\e  region 

It  Is  xunr  easy  to  nhov  that 
(lX.lA)  q^tteal  filler 

171 


d^at  -  [f^(t)  - 


*1^  •oltttion  of  thm  TvriAnce  equation  vlll  cater  tfee  resion 

(lfc.l3)  at  SOM  tlM  k  t^.  It  pjfflerB  to  consider  tlie  bslwnrior  cf  the 
optlanJ.  filter  after  t^-  ^  Introduee  the  Igaimacnr  fiittetlon  [ih] 


7(i,,  t)  - 


1  ''2 

Ou.M  " 


(14.15; 


K3d  TCrlfJ’  ttSKt 


.  y(x,.  t)  .  - 


1  ''2  . 


lA  otlMr  vords,  Y  Is  unlforaly  bounded  froa  above  azul  belov.  derive 

tive  V  of  V  ailong  skotlons  of  the  aptlml  filter  1«  given  'oy 


dt“  ' 


4o>,,/dt  dx,  1 

-  - 1  ®  St^  ^ 

“  *1 

*  ~  [— —  ]v^  , 

+  «  ^5 


(14.16) 


idxen  t  t  t^^ 


ifalch  ah£rs3i  that  V  1*  atrlotly  nAjcatlve  vhen  t  h  t,  urlLe!:''^  xt  0. 

.i.  jk 

This  prores  (l4.l4),  in  riAv  of  the  vell-Jraoun  theorea  of  I^apuaov 

Ttmorvm.  1]. 

1h>  eon  now  ooeg^lete  the  qualitative  exiAl^io  of  the  variance  @q\i«,tion. 
lot  g^*^(t)  axi&  (t)  be  two  such  ■olut-'.oas  of  the  verlonce  equaticm. 

It  is  verified  tuat  if 
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ttea 


d8<r^(t)/dt  -  tf^^<t)  +  f^^(t)lito^(t),  (14.1T) 

>j(t)  «  f^(t)  -  4>(t)/r^(t), 


and  iB  daflnad  slBilarly. 

If  x)  and  “f)  •re  tha  1x1  tranoitlon  isatricea 

of  the  optiaal  filter  correapoodlng  to  and  then 


ae  la  inaedlately  verified  by  differentiating  and  uein«  (l4.l7).  Hence  the 
dlBtance  beteaen  any  tvo  eolmtlowa  of  the  variance  eguntlop  vtileh  start  at 
a^(t^)  h  0  tenda  nalforaly  to  taro  vltb  t  -+00. 

Equation  (i4.lfl)  la  the  obvioua  analog  of  equation  (12.IO).  An  before, 

WB  can  define 

lisi  djjU;  0,  t^)  - 

t  -♦  -00 
o 

as  the  moving  eaulUbrlum  state  of  the  veirlance  equati  on. 

A  partiotilariy  ncrtevcrthj  feature  of  the  lorapunov  l^jmctloxi  (iH.l^)  la 
th/it  it  provides  a  quajititatlve  TBe^sure  of  the  fact.cr&  1  nf j-u^nclng  tJtr 
etabillty  of  tbs  optlawl  filter.  Bxio  nay  be  sser.  from  the  bracketed  tare 
In  (l4.l6)  uhlch  conbalna  the  tvo  ratios 


end 
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(14,19) 


IChiB  first  of  these  Is  Jiist  the  MBS«4{e-^o>nolse  ratio 

rar  [y^(t  Jt)  J/rarCvj^Ct)  ] 

of  the  error  signal  £^(tjt)^  of  the  optimal  filter.  lEhe  less  noisp  is  the 
error  alsoel;  the  norm  stable  Is'  the  optimal  fiii;er.  ISie  second  ratio  In 
(l4.19)  is  a  SRMumre  of  how  effective  the  optimal"  filter  is  In  counter¬ 
acting  the  raodciffiiass  Irtrcduced  hy  the  ^^Its—ncise  p^oce  ss  Beth  ratios 

can  be  related  precisely  to  Informatioa-theoretlcal  concepts,  as  Is  diiscw»3ed 
eliseVhere  [2$]c 

If  the  oonplexlty  of  the  problem  Is  increased  Just  a  little  more,  the 
discussion  qf  tswan  the  steady-state  properties  of  the  rarlenco  equation 
becomes  tpilte  InTOlTed,  A  good  illustration  of  this  state  of  KffairH  is  tlic 
followlns 

(lU.20)  S£Am?X£.  Qib  model  of  the  randem  process  is  as  shown  in 
fig.  O  A.  lx  oonsisxs  of  two  constant  first-orucr  systenui  whose  outputs 

are  observed  separately  In  the  presence  of  independent  >Jhlte  noise;  the 
coapllcatlons  which  arise  are  due  to  the  fact  that  the  random  inputs  v^, 

a. 

Wg  may  be  correlated  —  this  intix>duceB  an  "interaction"  between  two 
first-order  problems. 

*Q)e  matrices  corresponding  to  fig,  are 


Since  the  aioasureinemt  noises  independent,  we  must  set  »  0.  Ihe 

optima],  filter  Is  shown  Ir  Fig.  9B. 

Vfe  HSDVHoe  of  course  that 

r^l>0,  ^22^°-  (14.21) 

^  This  problem  was  suggested  by  R,  S.  Bucy, 


A  M  ft 


*12  ■  f^’ll  *  ^2a)'’l2  * 


*22  ■*■ 


^U^I£  _  ^12°22.  .  .  ( 

'll  '22  ^ 

^  ) 

-  +  q _ _  / 


(i4.22^ 


Ife  shall  inrestloate  only  the  prbblaait 
Llhrlua  stataa  has  (i4.22)T 

^V)  slHSplify  the  aetatica.,  va  sot 


Its  In  ItiJ 


^*12  “  ^22^^22/- 22^  j 

>^11  ■  ®ljAu'  ”  '*12^  '^21*'22’  •‘22  ’  ^ 

“1  -  •‘U  '  “2  -  >‘22  *  4-  J 

Letting  the  he  eq:iial  to  sero  and  tmlng  these  ahhrerlstlosxe,  (l4.22) 

reduces  to 


("ii  •  ^ia>®  ■"  *12  ■  “i' 

^“u  *  *22  "  ^11  ■  ^22^*12  “  >^12' 


(*22  -  *  4,  -  *2 


(l^^.2l1.) 

(lA.a^) 


(aA.2<6) 


In  addition,  g  must  he  a  ncnnegatlTS-doflnlte  mtrlx  so  that 


Poo  **  0, 


(1^26) 


-  PiiP2g  -  ^  0. 


(1^-29) 
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It  vlll  ba  oontenlent  to  utilize  an  auxiliary  relation  idxlch  le 
obtained  as  follow.  Md  (14.24)  multiplied  by  to  (l4.26)  multiplied 

by  asd  then  eubtract  (l4.25)  multiplied  by  Spj^,  Ibis  gi-res 

[p-j  1  Pgg  “  p)  “  ^*11^22  ^12^12  ^  ^22^11^  (l4.50) 

ifhich  coTi  alao  be  obtained  by  setting  d(dBtE)/dt  equal,  to  0, 

!3be  discussion  nov  proceeds  by  canslderlng  numercus  speci&ji  caseu. 

f 

Case  I.  »  0.  3y  (14.25)#  this  can  happen  only  if  =*  0.  Then 
equations  (14,24)  and  (14.26)  are  decoupled  and  the  problem  le  reduced  to 
tw  eepaitite  first-order  problems  ifclch  vere  discussed  in  detail  in  Exaiiq>le 

(l4.l). 

Case  XI.  /  0.  Bum  by  (14.27-29)  >  0  Pgg  ^ 

uov  ve  must  consider  several  subcases; 

Case  II-A.  0#  det  g  -  0.  JEy  (l4.50)#  this  iBq)ilQB 

-  ^12  >^11  ■ 

therefore 

(h^P22  +  MaaPu.)®  -  'h4<>L  ■ 


and 


'^ll'’22  ■  “  ~  *^(^'*^li)PxiP22  * 

•uhich  Is  poseibie  if  and  only  if 


det  jt  *  0, 


(14.52) 


•nd  (using  slso  tlis  fact  tbat  0)» 


lit  .  IS*  .  iljg . 

"u  *82  *12 

Suibstltutlng  into  (14.24-26),  vs  6brtal;i 


(14.53) 


*U  *  *22  ■  “u  "  *ll/*U  ' 

*11  *  *22  "  ■  ^2t  ■  *12/*12’  { 


(14.54) 


^11  ^22  ■■  *^22  ” 


this  aho"  -bheit  dat  p  -  0  only  If 


■f"  m  T  . 
"11  ^22 


(14.. 55) 


Thus  eOJ.  thrw  equations  (14.24-26)  reduce  to  one: 

^11  ^  **22  ~  ^11  " 

Bwiro  are  now  assiin  two  subcases: 

Q*ee  XX-A-1.  /  0,  dot  g  «  0,  ■■=^  Q,  TuiB  Cfin  .lv:; 

•  Ug2  “  0.  Then  (l4r24-29)  le  now  equivalent  to 

^11  ^22  "  ^xi  ^  i  ) 

I 

^11  ^7‘7*  ''  [ 

2  ; 

^3.  .'*^  2?  *"  * 

Written  out  explicitly^  thene  :: 


0  *  \p^\  <t^, 

Tfeei«  are  of  courso  nany  mtrlcos  ^Ich  satisfy  (3A,j57.).  For  inataiics. 

If  ^11  “  *^12  "  ^11  "  ^22  “  matrices 


all  a.’n'<?  7or<'  detcmiiir.uit  emd  all  arc  equilibrium  otates  of  the  voi-iance 
eqvMiLicm  (li+.22). 


C’i*.'/:--  p.,^  ^  0,  det  p_  >-  0,  f  wr  Ihezi  alizo  p,  ,  '■  0  atvl 

P,,,.,  ■  ('u  'We  cai.i  ^>1. p,  „  oiicl  p,,^,  .fT’r.'m  •">(;}  vi',.!i  t.he  »i.  Pi 
~>T  (,pz.  '1)  ■••j.nci  lioivf'  the  rt* sul ill's p.  •iutt.d.rvi.tic  >. •  .ju,  Kems'mlx-fi.za'  ii'.at 


SlmUarly# 

r^(«o)  “  oi  +  0^  a*oP  >  0i 

the  9qv®jL3-'ty  sS^pDi  Is  ruled  cut  here  d^ift  ic  the  faet  that  ^  0  hj'' 

(1H.I10). 

Ssshstitistlng  Isto  (l4,Vl)  aad  Ijettlng  -  ±  w  get 


<^11  *■  ^11 
^22  “  ^22 


<14.44) 


Va  noir  hare  to  cheeJt  idiethor  p^g,  and  defined  by  (l4.h4) 

and  (14.1»2)  actuaJLly  satlafy  (ill. 25).  Since  (lk,kQ)  fixes  the  slfpn  of 
ne  here  to  ccnuiliiier  2^  cases  corresponding  to  Tarioufj  signs  of  and 

Cg,  tie  can  lansdlately  rulo  out  eo«»  of  these  cases  by  noting  that  Cl*^.25) 
Is  equlTSlent  to 

and  reaemberlng  that  >  0,  Og  >  0,  p  i  0. 

If  •  1,  then  the  only  possibility  is  “  ^2  *  Moreover,  it 
Is  easy  to  verify  that  and  Pgg  given  by  (ll<-.l|l|)  oro  alueys  positive 

In  this  case. 

If  ®2  ®*y  fclloving  values,  ^rlth- 

out  violating  any  oteyiOMS  condition:  (l,  l)j  (-  1,  1),  provided  that 
■  p  ard  Og  >  pj  and  (l,  -l),  provided  that  (Xg  »  p  and  >  p, 

lov  wa  turn  to  the  eonditlon  det  g  >  0.  Wa  w«int  to  prove  that 
t  »  -  1  IS  the  ccly  case  ^Awre  this  is  true.  In  other  wrciii,  w 

eant  to  prove  the  ineijualitlcs 
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[r(i)fj^  nUr(l)fgQ  +  Og 

when 

«o  •  2*? 

[T(‘i)t^  *Ox  -  PHr( -i)fgg  +  0^  - 

when 

ie  0^  **  ^2  ** 

wham 

*0  "  *1  "  “2  “ 

[rC-Df^j^  -  s 

when 

5  =  -1,  €,  ■  1,  e*  «•  -1, 

0  1  -2 

tts£s  isi  iis  trss.'tsts  ssxst  iS  slos^  posit i^.  JtU  theso  li3««^iall~ 

ties  ftre  Uqpllod  by 

f 

«of«l  «o^  ’  *0^  Tr{«o)k22!]}  > 

jjgjwnding^  «•  obtain  tbo  tt^piivalant  iMqtiallty: 

r(«o)CP  ^  ‘r/^^  ""  l^aalK  "■ 


In  Yloif  of  (la.  tbn  Xeft-^iand  aide  le  ncsmegstiya,  S&e  parecsdlng  die- 
(nxaalon  shows  that  the  rlidht-hand  side  Is  also  nooinegatlve.  Squaring  both 
sides  wa  Obtain  after  bomb  calcuJLatloc  the  following  equivalent  relation! 

S  -  r^(det  a,)  t  >  0  (H^-^6) 

2 

Since  T  >  0  and  >  0,  th*»  inequality  will  bo  satisfied  ii  either 

detii/O  i^uJ  ^ 


Hence  we  distinguish  between  two  subcases! 

g^_n~Brl.  /  0,  det  P  >  0,  |f^|  r  \*^\  det  ji  >  0. 

Then  the  caaly  possibility  is  *0  *1  "  ^2  " 
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Pig V  Of  <l«t  g  >0,  dtt  ii  a  0,  and  |f^|  »  \^22'\’ 

Jf  9,  "  -^f  t)ttain  doaaldarlng  all  t^nrsa  omea  an  •««  tiiat  5  >  0  in 

W 

iflJv.wV  alfSars  ijeli^ia  det  o  S  0.  H«n^  tfiia  f»ae  cnnnot  ariBo* 

If  *  1  and 
o 

^11  (i^.^7) 

and  than  dat  g  >  0^  to  that  thl»  eaae  la  paaalhlo. 

Va  nov  eoUaet  all  reaulta,  and  atate  thana  in  terms  of  the  matrix 

If  dat  u  >  0,  than  w  muat  hare  Case  I  or  Caae  II-B-1  hecauec  of 

a^^.5a). 

If  dat  14-0  hut  /  0,  then  /  0  by  (lk.2^)  and  wo  must 
have  ease  ri-»~l.  Case  n-A-il,  or  Case  II-B-11.  If  "  ^22  ^ 
both  of  the  last  tao  caaea  oinjiLi  arise.  For  eatccnple,  take  »  2, 

,  ^11  "  ^12  “  otg  *  8,  p  «  6,  r(l)  - 

t(-1)  «  1.  Sttbatlttiting  into  (lU.?8)  respectively  (lU.l^)  and  (l4.42),  w 
find  the  following  two  nonnegative  definite  equilibrium  states  of  (lJi.22): 


'l  2“ 

when  a  -  -1,  and  1 

*21  2  ' 

2  4 

°  5 

?  24 

vhen  e  a  i. 
o 


If  det  |i  -  0  and  »  0,  we  have  either  Case  I  or  Caso  II-A-i, 
because  Case  .H-B  is  ruled  out  by  (l4.ilO). 

Collecting  our  findings  and  recalling  the  roeults  of  KwMapte  (l4.l), 
the  following  picture  oaerges  cor.cemir'g  the  equilibrium  otateo  of  the 
variance  equation  (14-.22). 

(l4.48)  !i5aEOHS!ML  (A)  If  det  ^  >  0,  (l4.22)  has  a  vnlque,  positive 

definite  equlllbrlm  state  £; 
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J 


“12  " 


^11^22 


+  +  f2  +  2 1^77)^77 )  - 

'  ^11  'u  ^  ^  ’^11  ^  =^22  ^  =^U^22 


®n”'ii 


'n* 


"U  ■  ^  r 


O 


^11^22 


(l*i.J»9) 


^22  -*’22 


f22+  /^T 
“  ^22 


- 


o 


12 


‘22 


The  f oraulafi  fy  r  5^,  and  5^  redx>ca  to  (.14.5)  ^  =»  0. 

(B)  If  dat  3  «  0  rad  ^  0,  there  are  geveral  pogslblUtlesi 

(l)  K  -  /  fgg/  there  io  a 

unique  aqullihrltaB  state  glveii  by  (14.49),  ^fhlch  jg  naa»ilnjtula.r. 


(2)  If  f^  -  f^  *  0,  there  la  a  \foique  fcquUJlhrlWM  state 
glvep  ly  (14,38),  vhlch  la  alnm»Ier« 

(3)  If  0  <  ■  fggf  then  tb.tfre  ara  praciaelT  equtll^lxai 

fftatea,  cae  eixigula-r  and  given  hy  (l4.38),  gpe  nonelogular  rad  glren  hiy  (.14,  49). 

(C)  If  q^  «  qg2  ■"  ^2^2  "  th<m^  the  foUcrlwg  poaallallltica  turlce: 

(1)  If  the  ocodltlgn  f =  f 2^  >  0  docs  nyt  hold,  twj  aeeeggarily 
0.^2  “  ^  and  the  prohljam  reduce*  to  ^mecngied  |%ret-<gj;^ 
mr  ha  (i)  gR8>  (li)  vjsi.  or  (111)  fora  egollj^TOi  arbata#  daTag>dl);^._m.^y«rfe^ 

(1)  f 3j^  <  0  BEd  f 22  St  0/^^  >  a  or  t.:^>0  C— 4) 

^11  ^  ^22  ^  ®* 


(2)  Jf  0  <  -  fgg,  thon  there  are  InficitfOly  nmicy  eguilibrlwa 

s-Uteg.  glren  lor  (l4.37). 

'UiB  have  seen  that  a  ccnplete  dlectiaaloon  of  even  the  eteady-atate  hehavlor 
of  the  vRjrl.enc*  e<SuetjeM  in  tAdfoua*  Vhat  ha*  heen  galnadt 

First,  the  results  prevldB  a  cfcselc  on  the  theoretES  of  Sect.  l6.  Second,  the 
various  apecial  cases  vhieh  arise  serve  ae  a  vemiag  that  strong,  general  re¬ 
sults  can  he  obtained  only  under  restrictive  condltiona;  ve  see  that  the 
hTlibthtses  of  the  tbeoress  of  8eot.  l6  cam\ot  he  easily  relaxed. 

Ifith  the  infoxnatlon  nov  avalleble,  one  could  actually  vrlte  down 
explicit  fon&iias  for  the  aolutlonr.  of  the  vairlance  equation.  IThe  erv 

eleaentary  hut  very  •tedious  and  little  Insist  would  he  gained.  A  nvDorical 
lUhstz^tlou  of  the  dynaw>lca.l  behavior  of  the  variance  equations  Is  provided 
by  the  next  exas^le}  a  ease  where  the  solutions  of  the  varl.anc?*  eqtjationB  can 
he  esxmBMd  in  eloaed  form  occurs  in  Kxasple  (l4.^). 

Bie  vezy  casqtlaxlty  of  the  preeent—  relatively  eliaaentary  —  exsaple 
ShOKB  that  a  djatalXad,  analytic  discuss  ion  of  the  variance  equation  Is  out 
of  the  question  for  higher-order  systewi.  We  ixust  therefore  try  to  clarify 
the  qualitative  properties  of  the  variance  equation  by  abstract  wethods. 

See  Sect.  l6.  Once  the  qualitative  behavior  Is  well  understood,  it  is  easy 
to  obtain  n^aierical  answers  by  mochlpe  cowputatlon, 

(iH.JO)  BZAMPIS.  Consider  a  dynaaical  system  In  which  the  acceleratlcn 
is  id&lte  noise.  Suls  situation  occurs  frequently  In  guidance  problems  (see 
alao  the  next  Ixasqile).  Vs  o»sua»  that  both  the  posltic®  and  the  velocity 
of  the  ey?t4MB  oan  be  obsexvedi  these  observations  are  ccutawinatcd  with 
independent, additive,  white  noise.  See  Fig.  9A. 

Ftob  the  figure,  the  defining  matrices  are 


and  2 


0 


1 


®»e  OlptiMRl  filter  is  shown  in  Fig.  J®*.  The  variance  squationa  arel 


tou/«  .  a»j2  -  -  i4»L/'ae- 

2  2 

'SCgg/dt  -  “•  “  **22®fa^*‘Es  '*’  ^ii* 

The  optliftl  c;elxia  are* 

t 

kjj^(t)  I. 

y^(t)  - 

k^(t)  -  h^c^iz)/T^ 
k^(t)  «  h^c-y 

“  hppgp^('t» )/rpp» 

W»  have  asBuned  of  courae  that  r^  >  0,  >  0* 

If  /  0,  then  the  variance  equationc  will  have  at  least  one  equlii- 
hrl«n  atate;  axil  if  furtheimore  >  0,  the  e<iulll'brlnai  otate  ¥lll  be 
unlejua.  Theaa  fecta  follow  frow  ®jeo3pam(l6.i8). 

Introducing  th-;  «^x:«vlati<^8 

"  >^22^ 

it  ia  easy  to  verify  that  the  equilibrium  Btate  Z  given  by 


Is  positiTB  definite*  If  "  0,  the  verlance  equation  baa  mo  eaulilbrium 

state,  unless  «  0  also. 

Bxe  solutlans  of  the  variance  equatlcn  were  ccoiputed  nvin»ricaJ-ly  for 

tM>  sets  of  values: 

« 

Case  A 


Case  B 


“22  ■  °5 

'll!  - 

■■u  * 

^22 

o 

1 

H 

0^2(0)  « 

■’ll  ■ 

h  «  2  ' 

’u  ■ 

Til  -  16, 

^22 

H 

1 

0 

t* 

”12^^^ 

=22(0) 

The  eolutlons  of  the  variance  equation  are  8hf*wn  in  Figo.  iO  arxd  n  and  tho 
corresponding  optimal  gains  in  Figs.  12  und  15-  St-ep  responRcG  of  the 
optimal  filters  api)ear  in  Pigs. 
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It  1b  clear  that  the  avallahlllty  of  a  relatively  accurate  velocity 
signal  greatly  reduces  the  error  and  speeds  up  On  the  other  hand^' 

l8  actually  someirtiafc  slowed  down  by  the  addition  of  the  velocity  signal. 
®hi«  phenonenon  can  be  explained  easily  by  looking  in  detail  at  the  verleince 
csjuatlons . 

(14.52)  ECAMFIE.  ye  shall  consider  a  data -smoothing  problem  en- 
countered  In  deteralnlng  the  position  and  velocity  of  space  vshicioc.  ITiils 
vSJLl  provide  a  convenient  Illustration  of  the  hamlltonlan  techni-q^ac  foi.  obtain¬ 
ing  solutions  of  the  variance  equation.  Moreoever.  the  esample  show,  hov  to 
obtain  the  teede.!  of  the  mn-iou  process,  directly  ixom  ptoysieal  eonsidcratlcwss. 

Bie  physical  pictxire  is  as  followa.  The  position  of  a  satellite  is 
measured  by  means  of  a  radio  signal.  It  lo  assvaaed  that  the  meaBure!»«snt  con¬ 
tains  additive  noise  which  may  be  taken  to  be  approximately  gausslan  ard  'wislttr- 
relati>^  to  the  bttndw:ldth  of  the  satellite  motion.  A  second  iBeaeurement  of 
the  satellite  motion  is  available  from  an  accelerometer.  This  reading  Xu  alco 
subject  to  noise;  but  here  the  noise  is  due  to  drift  and  other  very  elovly 
varying  effects,  and  aay  be  considered  to  be  a  constant  random  variable  dirring 
thr.'  inter.'al  of  interest.  ®hc  motion  of  the  satellite  is  llrearlted  and.  asBuroed 
to  be  one -dimensional,  and  stibjoct  to  a  constant,  gausulan  random  acceleration, 

The  problem  is  to  dselgn  an  optical  filter  vhich  provif  ,^3  Luc  best 
mmnlng  estimates  of  the  i>oeltlon  and  velocity  of  the  satellite  bfJieed  on  the 
two  tyf^ss  of  cieasurcraont  iioiso  and  the  variance  of  the  acc^leratlor.. 

Ihe  preceding  assunqntlons  are  fomalised  by  settlkig  up  n.  laodel  for 
the  message  process.  denote  the  radio  B‘*gnal  and  a^  the  reevdlng  of 

the  acne lerofflo ter.  Both  signals  are  sr^graoeed  to  be  Isnc^wa  oxsntly.  The  equation 
of  motion  (llnGo.rir.r'd,  one-dlaenBlomi,  with  unit  caasn)  is 

x.|  »  a(t)  ■  acceleration  =1  conetanL  =  a 

whore  «  is  a  gauBSiai.  random  variable  with  zero  mean,  lihe  accelercHaeter 
twsaaures  a  plus  a  constant  gaussian  random  T*iriable  with  zero  isexn  (the  blats 
error  of  the  acooleroaeter)  b; 

*  This  problem  was  suggoated  by  a  paper  of  £.  L.  Peterson  [jll*  3^^®  also 
the  writer's  discussion  of  this  paper  (52 J. 


■  a  +  b 


2  2 

3jet  Ba  ■  r  and  Eb  »  r,  and  dofina 

a  0 


r  -hr* 

a  b 


Va  Introduce  tvo  nov  random  Tariablae  vhich  are  orthogori&l  to  sack 
other  (and  thus,  by  gausaianneaa,  Independent),  ^e  first  of  tbeae  la  exactly 
KtsOto  arid  the  eecond  la  to  be  ©atlaat'-i'; 


Then  the  equatlona  of  motion  are: 


*1  ■  V  *2  ■  * 


«!  tXj. 


liie  Bodol  is  nov  fully  deacribed  and  ia.  shovn  In  Fig.  20. A. 
apection  of  Fig.  20. A  vo  havet 


By  Irr- 


r  -  0  0  1  ,  g  -  1  ,  and  H  -  [1  0  0]. 

0  0  0  0 


(14.55) 


Bte  -variance  equatlona  are 


dOjj^/dt  .  20j5  -  o^/r^, 

d^ j^dt  .  ,•  «2j,  -  Ou'’ia/‘'u- 

'^13/“  ■  ”25  - 

dOjj/lt  -  2,35  - 

.  «,J  - 

d.,V«  -  - 


(14. %4) 


*  Ih  ^*5  *®d  x_  are  not  independent,  and  for  this  rewteon  res-  ’ts  zts^tsd. 

there  Huet  be  oorrected. 


IBS 


are 


The  Haalltonleh  eqxiatloiis  (V^) 


T? 

0 

0 

0 

0 

0" 

'^1' 

^2 

•"1 

0 

0 

0 

0 

0 

^2 

*3 

m 

0 

-1 

0 

0 

0 

0 

* 

h 

0 

0 

0 

0 

0 

0 

J’l 

h 

0 

0 

0 

0 

1 

0 

-2 

r 

v« 

1 _ 

0 

- 

0 

0 

0 

0 

f 

0 

1 

Pt 

..J 

(14.55) 


Tfee  tmnaltlon  aaitrlx  corraapondlng  to  these  equations  is  easily  foimd  uaiiig 
(4.9).  (The  sixth  power  of  the  Qsatrlx  on  the  rig5it.4iand  side  of  (l4. 55)  1^' 
*ero  so  that  (4.9)  l8  a  finite  sum.)  5510  result  is* 


V ' 

1 

0 

0 

V-u 

^  A^ll 

1 

0 

-tV6r^^ 

8(t,0)  » 

t^/2 

-t 

1 

tV24r^ 

t^/l20r. 

0 

0 

*  0 

1 

t 

t^/2 

0 

0 

0 

0 

1 

t 

i 

1 

0 

0 

0 

0 

0 

1 

(14.56) 


l#e  asiuwB  that  the  initial  value  of  z{0)  is:  ct2n(o)  «  0, 

wlJlle  “  p  li*  the  effect  d;-;;  vO  the  bias  In  the  peading  of  ttie  aoccicrcuueter. 

(Of  course,  all  off -diagonal  terms  of  5(0)  are  aero.)  Substituting  (l4.55) 

Into  (15.12),  we  find  that  the  solution  of  the  variance  t:q>i£:tlon  oorrcuKOjid  i -■? 
to  these  initial  conii.ltionB  is: 


S(t) 


^11 


t^ao  + 


tV4 

t^/2 

t^/2 


t^/2 


o 

+‘*/2 


»Ao 


(14.57) 


It  iB  easily  verified  by  direct  eubstltution  that  this  Is  Indeed  &  solution 
of  the  variance  equation  ^ich  Batisfies  the  initial  oondltiLpnu  stated  above. 

nie  optimal  time-varylnv  gains  can  now  be  obtained  at  once  frt)m  the 
relation  5(t)  o  S(t)g'R“^;  they  are: 

k^(t)  -  t'"/lwKt),  “  t'Vay(t),  kj^(t)  «  tV2a(t);  (lH.5<S) 

whew 

a(t)  «  t^/2C  +  r^/p. 

f 

The  detailed  block  diagram  of  the  optimal  filter  is  shown  in  Fig.  20. B. 

It  should  be  noted  that  the  signal  u^  enters  the  wessage  process  and  the 
model  of  the  message  process  Inside  the  filter  at  exactly  the  satae  point.  This 
follows  from  the  fact  that  Is  a  known  constant,  indopondent  of  the  other 

random  variables. 

ae  differential  oquatlocs  of  the  optimal  filter  can  he  read  off  by  in¬ 
spection  of  the  figure.  They  aze; 


A  , 

dx^/dt 

-  t^/ka 

1 

0 

^^/dt 

=s 

-  t^/2a 

0 

1 

_ 1 

»  t^/2a 

0 

0 

(1^.59) 
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dw^/dt 

.  It-  n 

'  t  /ka 

-  /Ma 

-  t^/kx 

'^l 

t^/aa 

0 

dVg/dt 

Mt 

0 

0 

1 

^2 

■f 

0 

*r 

-  t 

1 

i 

0 

0 

0 

n 

L 

1 

L  J 

!]^e  transition  satrlx  eorrespoMln^  to  this  equation  Is 


f 


(t.T) 


0/a  -  6/2a  -  (r  -  8t/2)/c 

0  1  t  -  T 


(lJt.60) 


where 


0 


0(t)  «  t^/20  +  r^/p,  r(t,T) 


1  I 

6(t,t)  -  (t^  -  t5)/6. 


Thus  the  transition  natrlx  corresponding  to  (ll»-.59)  !-■  '01101  to  he 


(t^T) 


1 

a 


a-Ct^/2 

-8t 


(t-^  )a-(T'-fi^)tV^ 
a-(r-^)t 
-(T-®r) 


(T^/2-tT  )Qrt(  0^  -e-r  ^2)  t^/2 
-at  +  (P+TT  -  8T^/2)t 
p  +  T-r  -  iyr^/2 


Be  iir4:'.5lsc  response  cf  the  o:?tls»l  filter  relating  to  Is  gtwen 

„  ^  ■  -  - - -  ^  t  i  T. 

2{t^/20  +  r^,/p) 


The  impulse  )fesponsa  relating  to  sl,  is 
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fjw  expxvaalens  for  Pn0(t)  «ad  g.  (t,  t)  agree  with  tboee  gl^sn  by 


jravoriiOii  l5aj*  aowci'er*  ns  ootalns  r 


64(t>T)  »  y  1  - 


20r.,/p 


tat, 


{1^.52a) 


even  thotigh  the  two  laipulae  responMS  aze  not  the  ■oim;  either 
anower  la  correct.'  Bila  la  dvji  to  the  fact  that  ve  need  to  consider  only 
conatant  algoala  a^.  da  convolution  integral  foraed  with  either  (14.62) 
or  (lU.6aa)  gives  the  aaoB  answer  when  applied  to  a  constant  signal. 

de  comrentlonel  treataent  of  this  problem  (see  ..eterson  [51]  'sdxo 
follows  Shlnbrot  [8])  stops  when  the  lapulse  responses  relating  the  tssasure- 
oents  (z^  and  a^)  to  the  desired  eatlutes  (say,  x^}  are  obtained. 

Ihis  is  not  cooqpletely  agtiafhetory  because  the  ii^pulse  respui  aay  be 
difficult  to  rwO-lRC  physically.  Tor  instance.  It  nmy  be  easier  to  build 
the  filter  In  Fig.  20. B  (where  only  the  gains  tii«e-^iaryiRg)  than 

tS  l«ipie*©av  the  lairalsft  responses  (l4.6l«62). 

Sote  ftlBO  that  If  ouiy  gj.(t,  v)  is  desired,  it  can  "bs  realized  by 
l>dlaeDSional  dynandoal  s;/Btea 

dWj^t  »  -  (t^/4cE)Vj^  +  (t^/as)*, 

A 


i91a 


(i4,63)  RMMPIJS.  Conaiaer  the  third -order  Byntom  defined  hy  the 
oatrlees: 


Oie  modal  coneiet*  of  two  cjaacadod  luftegrfttom,  preceded  hy  &  alzi^tlc-Xag 
elaoent  with  traaefer  function  l/(«  - 

dCTjj^/dt  «  2tf^  - 
ddj^/^t  »  *  °22~ 

de^j/dt  -  +  «23  - 

ddgg/it  ■  togj  + 

"  ^?5®23  ^3  ~ 

dojj/dt  -  ^ 


Aneunlng  >  0,  It  foUove  that  the  steady-citate  toIymi  of 
la  gl-ven  hy  the  qnartlc  equation 


^uAu  ■"  ^35 


1/2 


9m  reaainlne  can  ha  ncnr  eaally  daterslnsd  ualsg  (^2h,6k),  Xf  f^^  ^  0^ 

theti  equation  (lH,63)  haj  a  unlquki  eolutlonl  if  f^,  >  0.  then  there  era  two 
aolttticsia,  one  of  Which  la  ruled  out,  howerer,  V  the  nooaagatlTwieaa  requlra- 
nmt  on  £. 


1QC 


A  dert&il«A  sBUytlttAl  diaeuMicKi  of  tlain  fwailo  trowld  Ite  'pointlota* 
Ibataad^  m  4lS(iit  roly  on  tlM  tteosPMHl  to  1m  atwtod  ifi  Ssot.  16  for  a  <|aallta<- 
tiYC  UB^ritaadltis  of  tim  btiiairior  of  (lk»6k). 

On  tlw  iHUBio  of  tfeM  axanq^les  diae^sftd  In  tiii9  hioction;  It  lit  e(mj«ctu?«d 
that  tna  hohnvior  of  th»  Tfy?l«ioo  a^iatloii  oan  V*  a»o«rtal*»«J  by  elewsntary 
(tbou^  not  oIhi^Ia)  al^oabralc  «Me!S  vlMnOTor  thd  stodel  of  the  procogn 

18  «t  leaot  of  th«  soeond  ardor.  Sy  Isaaple  (it.  65)..  this  Is  no  Icsn^r  |>o@3l'ble 
nfaan  the  »odsl  i»  of  order  three,  unl««i«  the  of  the  haallt^Mil^sii  •«;*»- 

tlons  Is  nllpotent  ae  In  Bnus^la  (it.^O).  5icr>J^  e  practical  polit  of  rletf; 
such  al^brsdc  Bsathods  are  uoelass;  one' should  resort  to  th«  digital  costpuier 
for  n\m0rleal  anseera  and  tc  Bom  a&ranced  aaalyala  for  vindsrstosdi>jg  thu 
qveilitatlve  behavior  of  the  optiswl  syateas. 


8  ' 


I*?? 


15.  MlBiHiA»T>rl«noe  mblMed  attlnaj-tlon^  Steie  section  ii  prepara¬ 
tion  for  the  detailed  tbeozy  of  the  varieuce  equation j  at  the  same  tli^e,  ve 
estahlieh  Intereatlng  connectlonc  hetiesen  our  methodji  and  the  classical  theory 

of  paraseter  oatiwation  ls%  OMi^ters  52-3^]. 

• 

yh  shall  study  only  the  caw  of  continuoua  tJisKs.  This  viU  a?pr®- 
clfchly  alBpllfy  the  foraulae.  !Bie  case  of  discrete  tine  differs  only  in 
trivial  details. 

Consider  a  special  case  of  tha  model  (1^): 

£(t)  -  K(t)t(t,  T)x(T)  f  v(t),  (15.1) 

ehere  5(f)  is  Bane  fixed  hut  unknown  state,  ®(t,  t)  ie  the  trEsnaition 
aatrlx  corresponding  to  5(t),  and  v(t)  io  a  gauastan  vivitc-nolss  procesD 
with  cov[v(t)]  ■  R(t),  \h  assume  that  R(t)  is  nonslngular. 

Given  that  *(t)  has  been  observed  in  the  interval  t  s  t  s  T,  we 

o 

elBh  to  obtain  the  lacst"  estimate  of  the  "pareoceterB”  x(t).  Uhls  problem 
MS  discussed  already  In  Exatqple  (9.3).  We  may  think  of  (15.I)  an  represent¬ 
ing  noisy  observations  on  a  free  dynamical  system  At  time  t^  it  la  decided 
that  the  "beet”  estimate  of  the  state  is  desired  at  time  T. 

Let  vss  consider  first  the  prcblsm  of  estimating  the  acailer  quantity 

IT  -  £’x(T)  (13.2) 

where  g  Is  an  arbitrary  but  known  vector.  Lot  ir  denote  the  estimator  of 

T. 

It  is  cleeu:  that  tt  will  be  in  general,  e  I'andom  variable,  since  it 
is  to  be  a  fiinctlon  of  the  observations  z(t).  Ito  define  the  ^beot”  esti¬ 
mator,  it  is  not  eufflcient  to  mlnlmlae  the  variance  of  it,  for  the  trivial 
estimator  whic’*  is  constant  (nonrandom)  has  xero  variance.  One  could  require 
that 


b«  a  ffilniiman. 


It  ie  more  eturtoiBiMy,  htnrsvs?,  to  rsssuire  first  that 


A 


E{Tr|x(T))  "  S.‘5(T)  = 


05 


ftnd  ttkon  «iniaii»0  (15.5)  subjsct  to  thi*!  constra-l^t. 


tn  a  mlniiiiaLjL^- 


Such  aa  estimator  does  not  aecessaialy  taclst  for  t!i«  acd©,i,  (2.5.1),  aa 
is  tBBiediatels'  olnrlotifl  setting  H(t)  identically  oqpial  to  aero. 

Evidently,  the  etjtistence  of  an  imhiaaed  eotlKatocr  la  a  characterlatlc  property 
of  the  oystaai  (ij.i).  2ils  Bkotirates  the  folloirtng  iaportast  concept. 


(15.5)  nBPXHmOai.  a  system  (l5.l}(or  (l^))  is  said  to  coi8^1s'>^ly 
ol)e!ami>ie  if  for  every  t^  there  exists  a  T(t^)  such  that  for  any  pom- 
»jte;r  t  ty  {15.2)  oas  can  constmot  an  unbiased  satlmtor  x 

vhlch  is  S/^function  of  the  ohaermtlons  £(t)  in  the  interval  S  t  i  T(t^). 


xhere  £(t)  Is  on  JUbitrsry  (at  leswo;  pl^rcevrlee  coEitteuouis)  veoVr  Tksic-- 
tion  cf  tiii»9 

It  aey  bs  that  this  property  hoCLds  fior  bos»  Inzt  not  ail  vectors  g  in 
(15.2).  .i  veetoar  £  ^Inh  Isy  (15*2)  doflww  a  peraaErber  x  fc«Tl?3«  i».  .jablarcd 
Asl.j.sB,tor  ^  is  aallad  an  ctbmcE^gsaba  Xt  t«3»®  ost  ti&t 

ia  the  dwsil  of  Idle  eonoept  Of  <«Xfttyo3Jial^lit^  ■ahlcii  is  briefly  sasat-losjed  ,16 

ssd  discussed  in  sa^ch  furthor  detail  in  [25,  5?]. 


Tor  the  pwiamt  ps£5«(ies,  im  dtewagterllMtlhre  of 
will  ba  anfflolent: 


<i)  Ltja&  (li) 


ffhSaaigs 
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®  1 

K(V  “  4  T)l'(+03  (t)i(t)i(t,  !r)dt 

1«  poaltlTC  definite  for  som  T  >  t^. 

proof  [33 !•  (i)  If  positive  definite,  then 

8®(t)  -  R'^(t)i[(t)t(t,  T)£  (t^  *  t  S  T)  (1^.8) 

f 

vlU  dsfljoe  on  voiblased  estJjasator  (1^.6). 

(il)  Siq>posft  -Hsat  systeai  (15.I)  Is  coopletcly  observable  but  that 
I*  singular.  Tben  ttere  Is  a  vector  £  /  0  such  that 
^  j)  *  0.  Qlhen 

■^(t)  -  l(t)t(t,  T)£ 

« 


is  l<S«»tically  tnro  In  the  interval  [t^,  T]  stace  It  is  a  ctantlnuous  func¬ 
tion  of  tlae  end  since 


*<,  * 


<t) 


dt 


T) 


0. 


lov  let  a^(t) 


dafine  ar  unbiaiied  esti«Rtcr  of  e’x(T). 


Then 


0 


t  2  \  Xy  .  a  . « 

j  B  (T&;ax 


t 

o 


T 

J 


t 

o 


(t)l(t)t(fc. 


l)£  dt 


^dilch  coatrsdicts  the  hs^othssls  that  £  /  ft.  Q.  K.  D. 

Several  points  Should  be  noted  here. 

Sven  if  the  aatrlr  U  is  sinfulaur,  it  supplies  vrUAssbls  infomatlon. 
A  aodlficatlOD  of  the  preeeilrs"  proof  ahovtt  nejsely  that  y  Is  an  observable 

[t^j,  T]  If  and  only  If 


Is  -m(V  0) 


(15‘9) 


y^V9  !•  fisy  psoudo-iirr^rfM  of  ^  (s«e  ijp^nf&ls  A). 

Vhtle  (15.7)  !•  of  oontral  theoratloal  laport«ne«j  it  1»  not  oonTan“ 
lent  to  epply  in  eonczvts  .mmc,  iMbc^v^iCs  ^  is  dlfflcvtlt  to  c&lculntQ.  If 
sy«t«ra  (l^.l)  bM  ocnstMit  aa«fflciaatBf  t^£I  follovln«  purely  al^bralc 
criterion  1»  equivalent  to  (ll>*7)i 

r«jktK’,  r‘H’,  (y*)"'^’]  -  n.  (id.lO) 

aiiiB  1b  proved  In  [35l»  u*lng  (15.7)» 

Dlfferentletlns  the  Integral  defining  K  vlth  respect  to  T  leeda  to 
the  differential  equation 

dij/di  -  -  -  MrCt)  +  h’(t)r“^(t)k(t).  (15.11) 

If  K  baa  an  Inverse,  then  this  equation  tiecanfia 

dM"V<i5P  »  ||:(T)H“^  +  M~^'(T)  -  (15-12) 

which  Is  a  special  case  of  the  i^trlaaoe  equation  (lll„).  !2ius  is 

analogouB  to.  £.  ISiia  Is  eMily  seen  also  by  cooputlng  ths  covariance  oatrlx 
of  the  unbiased  eatiaator  ^(2)  d/afined  by: 

^(T)  -  ,  T)  /  fCt,  T)l'(t)5"-"(t)*(t)dt  (15.15) 

t 

o 

(see  (15.85).  We  find 

-  M“^(V 

E((s('f)  -x(2)]ix(T)  -x(t)]  i*(T)}  -  M'"(t  .x>.  (15-1^) 
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Bxoopt  in  the  oonstoat  oaMj  hownTor,  will  not  be  inrertibls  in  s^neral 
And  therefone  m  wuat  vuraaLLly  deal  vlth  (l^.ll)  rather  then  th::;  Tar  lance 

equation. 

[She  matrix  T)  le  ■well  Imovn  in  claeslcal  atatlptlcs.  If 

T(t)  is  gaussian,  then  M  is  the  Tteher  inrorBatlon  aatria:  [5^]. 
definition  of  the  latter  .  Is  aa  foUovs.  Let  fjj,($lx(T))  bo  the  corditlorial 
probability  density  functional  of  tho  observaticaia  5(t)  in  the  interval 

glTan  5(1).  (in  the  case  of  ccntinuoua  ttn»,  thie  is  a  probabi¬ 
lity  density  f)ff:ictlon  of  curwmt  s(t)f  the  risoroua  definition  of  vhich 
1»  Boeauhat  delioate.)  Hie  Fiaher  infonaation  Bw.trijc  ie  defined  as 

SV(s|x{T))  ■ 

^  ‘  Kj'  * 

In  the  case  of  gaueslan  noise  v(t)  ve  have,  ptirel/  fomally, 

T 

f  (dat(T))  »  conat.  exp[-  ^  /  i!i(t)  -  H(t)eCt,  T)x(T)f  dt]  {1%26) 

.  -  ^'o  ST 


and  one  can  check  easily  that  the  two  definitions  of  M  coincide.  Notice 
that  In  the  gauss lani  case  the  iiifonaation  laatrlx  is  Independent  of  the 
paraneter  x(T). 

Hie  most  iaqiortant  application  of  the  Fisher  Infonoatlon  matrix  lo  tlie 
feoBOiui  Cra8ier~Rao  or  information  inequality  [25,  Sect.  52.  10,  Sect,  7  ]. 

If  ?(T)  Is  any  unbiased  estimator  of  x(T),  'then  the  Information  Inequality 

.  * 

IB 

*(Cx(T)  -x(t)][x(T)  -  x(t))' ix(T)}  J:  (15.1T) 


which  is  valid  of  course  only  if  M  is  positive  definite. 

We  h&vf  Just  seen  that  for  the  estitBator  dofined  by  thu  equrJJty 

sign  is  actually^  attaiaad.  Assuming  x(t)  is  not  coastant,  it  can  be  ahovn 
[55^  §38]  ^hat  the  equality  sign  in  (I5.I7)  arise  if  wnd  only  if 


IT  symaatric  matrices,  we  write  A  >  B  [A  *  B, 

fact  that  A  -  S  is  positive  definite  [nonnegative  definite]. 


to  express  the 


n  nQ 


can  be  factored  as 


(15.18) 


JfliJUkS  g;  ^  b  gr^  ferbitrery  fiiPctlonB. 

Bfr  expanding  tbo  lutegrand  In  (15.I6)  .  we  sec  unsediateXy  tLjvt  this 
condition  is  true  in  the  gauaalan  case;  In  fact,ijaen 


Si(x(T))  »  x(t),  2b(x(T))  =  ,  T)  ' 


Vhenerer  the  probability  denolty  functional  can  be  factojced  ae 


-  g(»(0»  iWMO) 


(15.19) 


one  aays  that  x  is  a  auffleient  statiatic.  As  la  obvloua  from  (15.19),  in 
thlfi  caae  \  contains  all  Isaforaation  Mbich  the  data  |^(t)  (  the  obcgrrc^ 

Tmlues  of  5,(t))  coonroy  about  the  paraiaetor  x(!P).  This  explnina  lntuiti?eiy 
why  the  equality  algn  vould  hold  in  the  lnfom'.tlon  Ineqxiality. 

Since  the  (atrlct-aanae)  nlniJaal'*Tarlsnce  unbiased  estltaator  turns  out 

in  tha  ga\»alan  case. 

to  be  linear^  this  estimator  constitutes  at  the  saaes  tls»  tho  solixtlcai  of 
the  (vlde-aenae)  problem  of  finding  the  w1n1uta1*>arlance  linear  esttaator. 

(See  Sects.  2  and  10.  )  tfs  now  pronre  this  fact  by  wothoda  independent  of  tho 
preceding  discussion. 

(15.20)  OAnoo'4iAHS37«  ToEsiiSS'l-  Af?yume  th<»  proc^gg  fl^.l^i  is 
obserfable .  Let  T(t)  be  a  •wbite-nolse  process  (not  nacuiBWg'tly  gaamislaxi) 
vlth  a  nonalxigular  oorarlanoo  vatrix  Bl(t).  Ihen  th»  minijaBfvl  -^riancft  linear 
unblaeed  estimator  of  saar  par— ater  r  |[^'x(*r)  of  tarocess  (15.I)  is 
T  -  e'IC*),  Tjbgjge.  i(T)  iljJllCflSOL  (15.15^ 

Frooi.  Ifst  p  be  an  tmbiased  linear  estlBBtor  of  dafinad 
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1S»n 


T 

p  ■  /  r*(t)is(t)dt  , 

t  ~ 

o 

5?  P 

irmr  p  ■  gf  /  r‘ (t')'r(+.)dt} 

t  "  -  ' 

o 

« 

-  /  lli°(t)  -  (r(t)  -  .. 

o 


since  p  In  unMneed^ 


*  T  T 

/  r- (»)*(■“ )a°(t)«  *  /  r'(t)H(t)t{t,T)K'’-(t  ,T)Edt  .  ||eI|- 
t,  ‘o  i! 


Beace 


-mr  p  •* 


»  -nur  T  , 


Slnc«  R(t)  in  po€3.tiT9  dei’jLnlte  for  all  s  t  *  T,  tlie  equality  Big»i 
oaifi  occur  only  If  r{t)  •«  B(t)  «v«nn*t»re  In  this  Interval,  that  lo,  only 

If  p  «  r.  Q.  X.  D. 


It  onOulu.  he  noted  thint.  the  giHiJt'»-fi4U’aoT  tcQort^  cc 

requlro  the  ibssuaptisa  tlw.!.  t  he  a  ehltp-aolao  prwc«eB. 


In  fact,  if  Y 


hae  the  nonelngular  coYasriance  aatria 


eovIxCt),  t(t)]  -  5(t,  t)-^ 


aoo 


than  letting 


Sft/^  _  f  t  At/*  I  »\»l /*  t  Nl*"*/*  * 

m*-»  */  “  j  4  S  •  */*  x**  /s  #  w/ax^/xx •?*/“»  «“ 

*  4! 

’o  ’’o 


siaixftl  'fsriaikflio  3inaftr  VBibift»»d  estiBstor  of  ^(9)  Is  sIt^ 

IT. 

4(*)  -  !f*(V  *);  [/  l'(t',  *)J'{t')8'\t',  t)4t'l6(t)« 


Wft  nov  eonalAar  tbo  (jatstlon  of  ptsyslc&Uy  zmlizlxifc  the  v^i- 

MUMi  xsttbiJUMd  AStlMitor  x(7)  >7  Mans  of  a  dlTXiSffilcnl  S7»t9».  In  thise .  cosb. 
the  aneui^ioa  that  r  i«  a  thlte-aolM  proceea  if  &  very  a^spreci^blsi  sl^li  - 
floatlon* 


(15*21)  Bie  wtBlmlj  sartwce  xi^jAMd  eetlastror  |(T)  gtyan  1:^ 
(15*15)  la  the  teaiiaftl  t»te  of  the  .  . . 


(S^/dt  -  2(t)|  +  5(t)[*(t)  -  g(t)x]j 

xdiere 


K(t)  t)H’(t)5"*^(t), 


(15.22) 


B»  mtrli  block  diagnui  of  this  eqjiRtioa  is  Mentical  tflth  F5.^?.  5, 
which  irefors  to  the  pptlaal  filter  only  difrcreuoe  lies?  lu  the 

deflnitlcT?  of  tbi. s  !•*  rwiir  min  #Wf>  btAW4B  *4  1  X  W 

•  4  *  ■  '  ■  *  — '  .  -  • 

notoil  that  t)  la  fortaallythe  sojme  an  £(t|t). 

Proof.  Let  T(t,  t)  be  the  traincitlon  usatrlx  of  the  filter 
Flrat  xie  show  that 

I(T,  t)  ■=  t^  <  t  S  T.  (15.2?) 
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Sii«  fonnila  la  Itftse  if  t  »  ¥  becauao  thm  tiM  ri^t-haod  side  of 

(1^.23)  Jrddnoaa  to  tbe  xnlt  Matrix.  t).  regarded  sa  a  function  of  t 

vith  f  fixed,  ■atltflea  the  diffiara&tial  e^uatlcn 

a 

-  i(®i  -  !(*#  t)CE(t)  -  t)r(t)R**\t)E(t)i,  (15.24) 

ae  Is  saally  seen  197  dlffer^:1j»tjlri>^ 

1(5,  t)i(t;  f)  -  I 

f 

and  viaing  (4.8),  Differentiating  the  right-hand  side  of  (I3.25)  vith  respect 

to  t,  using  (15.11)  and  the  pseudo -inverse  lessaa  (A.4),  ue  verify  easily 

that  fl5.24)  holds.  Thus  ^  defined  hy  (15*23)  is  Indeed  tlie  transition  matrix 
of  (15.22). 

(9e  see  Ineedlately  from  (15.23)  that 

X(T,  t).0  (15.25) 

idiere  t^  is  the  largest  value  of  tine  eucb  that 

M(to>  tj^)  “  0. 

Since  a  transition  matrix  is  never  singular,  this  consegucnco  of  (15.23) 
is  of  course  absurd.  In  fact,  f(T,  t)  is  given  by  (15*23)  only  for 
<  t  *  Tj  fvtrther 


llm  f(T,  t.,  +  h)  *  0  ^ 
fa  -♦0  “ 

and  W6  sln5»ly  define  f(T,  t)  to  be  aero  for  S  t  i  ) 

In  View  of  (15*20),  we  have  to  prove  only  tvo  things.  First,  that 

x(T)  -  /  I(T,  t)K(t)z(t)dt  (15.26) 

t 
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is  an  unbiUMd  aitlBfetar  of  5(T)#  second,  tliat  the  ccmiriance  matrix  of 
X  is  T)« 

It  is  oasy  to  ta«  tl»at  x(T)  giTsn  by  (15.26)  is  unbiased  if  and  only 
if 

T 

/  l(’S,  t)K(t)g(t)i(t,  T)dt 

is  the  unit  atatrlx.  rising  (I5.II),  the  i»i'ecadlii^ii  iategi^  ‘beeches 

T 

;  t)i(t,  T)  -f  i(T,  t)*(t,  T)ldt, 


-  I  -  X(T> 
*  I 


because  of  (15.25). 
Further, 


T 

e««rrx(T)l  -  /  ¥(T.  t)K(t)R(t)5'(t)r (T,  t)dt. 
“  t 

o 

Ry  (15.22)  and  (15.25),  the  integral  ie 


’S),  f  »'(t,  T)iS(t^,  t)g*(t)s"\t)g(t) 

o 


The  psaiido-ln-rorse  lama  (A- 4)  shows  that  tlas  bracketed  ter»  is  eijaal  uo 
g(t^,  T).  Hence 

cotIXt)]  »  ir^(t^,  T). 

The  proof  of  (15.21)  Is  cosgtlete,  (Bssark  added  In  proof;  She  presedins  a*^- 
lasnt  shoirs  that  actually  5(t)  defined  by  (15.22),  is  an  \sibiased  estiaator 
of  x(t)  for  eTi  tit-  >  t  shire  t,  la  the  first  saloe  of  t  for  which 
Mft  .  t. )  Is  nena insular.  ^ 

’  O"  X'  -  ■ 
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(3^.87)  MDttIQL  Xbe  theorem  Just  prored  shcwe  that  tbs  filteriog  end 
UDblued  estiMktlOQ  problsss  are  goremad  hgr  sBeentinlJly  the  aame  tbsory. 
me  iB  *  Italliar  etate  of  affairs  In  the  eolaulua  of  Tarlatlons.  One  Male 
aqnatiom,  the  feallton-  aedbi.  partial  dlfftereatlal  e^tiotVj  corers  a  wide 
variety  of  problems,  the  dlfferenoes  hetveea  the  Tarlcaus  types  of  prohiemB 
heiJBg  rapresanted  hgr  the  honsdaxy  conditions.  Hhs  bsailltoa- Jacob!  partial 
dif!<6r«esti«l  acssticR  Is  equlTOlent  to  emr  ^rarlauCv  eqtim^tlca.  Iho  hoiaklary 
eonSlticu  kz  the  fllteriaif  case  is  that  £{t^H^)  is  sosa  narmegatlie  de- 
finite  as^trix.  In  the  unhlaMd  estimtion  problem,  the  initial  condj.tion  is 
5(to|t^)  "  00,  tlileh  is  the  mate  as  t^)  r  E^(t^jt^)  »  0. 

CXearly,  the  aolvtisui  of  the  filtering  and  cetijuition  problei&s  will  in 
flBBaral  ha  different.  Ve  can  see  with  reference  tc  Sxsmplo  and  Figs. 

IB  and  that  the  qptljsal  filter  Is  usually  not  unbiased.  In  other  uerds. 

If  the  signal  (t)  has  a  mean  ccaqpoeaent,  this  mean  cca^^xxient  will  be  ro" 

prodoeel  vith  sat  error  because  the  tmlt  step  in  t^Ct)  does  not  resiilt  In 
a  Iteit  stsp  in  ig(t), 

(l%S6)  mciK.  Isit  us  indicate  briefly  hov  the  wixijima.l  Torlance  un> 
blaeed  estlaetor  oen  be  computed  In  real  time.  There  are  essentially  fcAix 
poealMlltiea: 

(a)  In  the  most  obrloue  case  shown  in  Fig.  one  simply  per- 
x&ma  the  multlplieaticn  Indicated  In  the  integrand  of  and  them 

Istegxwtes  with  respect  to  time.  The  multiplying  signals  can  be  generated 
|y^a  dyaamleal  aystem  whose  Initial  state  is  taksn  as  «(t^,  T 

^  zvUmr  l!!COBnraiiimiit  if  analog  computing  efuipmant  .is  used,  because  of  the 
diflKteulty  of  aeemrate  natlpUoatlaa. 

(l)  Dsr  Inspectioo  of  (15.I5),  we  mrte  that  ±*it,  T)g' (t)5“^(t) 
way  be  intexpareted  us  tha  (generallasd)  tspulse  response  of  the  differential 
eipMctiOB 


djj/dt  -r  (tk  t  I*(t)8”^(t)a(t) 


(tlwi  fSTM  part  of  'MLoh  ic  tte  •fyolat  of  ««tl- 

aAtor  tsM  tte  oteioua  ptqrBloal  TWllMtion  itevn  in  ?!<•  21.B.  It  wasr  te 
ecBfwl«Bt  to  otenge  ^rarlaliliifi  in  sneh  n‘«qr  ttet  tte  tetris:  g  teoteta 
tte  Itentitgr,  Steh  »  ajptftte  cte  te  otutiJlj  tnilt  telng  itaadari  tealos  ooa-> 
poaa&ti  tet  it  bu  a  eoriotta  diteAraataga*  XT  |;(t)  dafinaa  «ui  uTB^toti- 
Ufftrcntlal  e^jatioQj,  tten  e<xnre^ponAliig  «Ajois.t  5?b>^ 

Ate#4  m  tMiMte**  Iv  teiawwilt  a«teteHkte^*l n  A  ^maa^k 

'■aa*  MaAW*..  ^1^  \  ^  /  woafc.****^  *«  ~  ~  —  «' a  »* 

this  mt^Qd  tea  noted  hy  i&iobMn  i;j6u 

(C)  Soting  tte  difficult^  Juat  tentiottad,  aeramX  avratere  (^>et 

proniscntJi^  Xugglaa  [3-71}  tern  auggaated  tte  foUcalng  nltesn&tlTS. 

tte  raoord  of  the  oteamtd  foactioa  B(t)  (t^  *  t  i  T)  la  Invarted  in  tlae. 

that  la  to  saj/,  ve  Intiroduca  a  nev  tian  vsrSahla  t'  dafinad  ^ 

t*  -  T  -  T  -  tj 

and  oonaldar  s(t')*  2.(2T  -  t)  instead  of  c(t). 

Since  4'(t,  T)  ■  t)3“^  la  tte  tracaltloa  tetris:  of  the 

differential  equation 

dg/dt  -  - 

of  (Iv.l),  It  la  clear  (by  changing  -varlablBa)  that  [e^(2T  -  t’,  T)]”^  is  the 
traaaltlon  tetrlx  j(t,  T)  of  tte  dvial  syatea  of  (l^) 

d^/dt’  -  F’(t’)x  +  H’(t')w(t*),  (15»^) 

vtere 

r(t')  -  F'(ZP  -  t). 

Hence 


O' (21  -  t*,  T)  =1  (f(t' ,  T)]’  .  *(T,  t). 
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3Su»rofax«  after  the  ahaage  of  terisbZed  t  ve  tb«i  phTsloal  reali- 

■atloa  ebeim  In  Tig.  filA. 

She  refuired  tl»i-lirrareli&  lay  be  parf«?wned»  for  Inetence^  by  recording 
g(t)  on  the  rttsorSer  and  then  maalag  it  beolriiarde. 

(d)  yinalljTi  tn  ha’^  the  reaUBatlon  pror^ed  by  Iheoren  (1^.21). 
It  should  mS  uoted  that  In  this  case  the  M5nl3al  rariance  mblaaed  eatlTeator 
Is  asymptotically  stable  (see  neact  seotlon)  no  tlMS-lrrrerelcxn  is  reqiulrod, 
wad  It  Is  not  neoassaxy  to  change  coordinates  so  as  to  oake  M  unity.  On 
tsi?  'ythsr  bend;  one  reqmins  tlne-rasylng  gains  E(t).  See  Pig.  2ID, 

(1^*30)  XCdUnUS.  She  slaplest  estlwatlon 'problen  coRcems  the  detection 
of  the  sine  save  in  vhits  noliK!!  ‘ 

r^(t)  ■  x^(t)eos(t- T)  +  X2(t)Bin(t  -  T)  + 

Shis  probloi  Is  of  tnportence  in  dynastic  testing  [3Q]. 

The  corresponding  astrices  are 


Bta  sntrljc  M  given  by  (15.?)  la  foxmd  to  be 


i!<V«  ■  5 


2(T  -  t^)  +  sin  2(T  -  t^)  -1  +  cos  2(T  -  t^) 


I  -1  +  cos  2(T  -  t^) 
L 


.  (15.51) 


2(T  -  t^)  -  Bln  2(T  -  t^)  ! 

J 


It  is  easily  checked  that  this  tgatrlx  is  positive  definite  for  all  T  > 

Appreciable  0iiig>lxiicatA.on  results  w«  take  advantage  of  the  oithogona- 
lity  properties  of  sine  and  cosine.  Thus  me  is  led  to  oasume  that 

T  -  t^  ■  qff  (q  n  positive  integer)  (15.52) 
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in  vtiich  ease 


M(t^,  T)  -  |<'P  -  t^)i. 

Let  us  8iY6  also  an  eopllclt  e^^presaion  for  the  tifleHTarfin^  gains  of  the 
KlnlMLl  farlsnoe  wnbtMsd  estinatoz'  Using  and  (1^.51) 

assiadng  m  find  ttet 


2(t  °  t^)  •*•  sin  2(t  -  t^) 

’’nhx  ”  > 

sla®(t  -  t^) 

_  o 


(15-53) 


^11^81 


S[(t  -  t^)^  -  ein®(t  -  t^)] 


AU  t^  the  aelaies  of  aad  are  hoth  Infinit^^  Shay  Saeraa: 

aicslly  tc  VS(^  ”  %)  0. 


Urn  hlock 


of  the  filter  is  shovn  In  Tig. 
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16,  Prgperblai  of  the  •varianca  e<Mfttlon.  Sh«  main  pisrpoao  of  this 
section  is  to  generalise  the  results  of  Sxeaiplee  (I2.7)  emd  (l4.11),  '«e 

do  this  Iqr  trjrlzis  ‘to  imitate  the  lestbode  used  to  study  these  exasplea.  !I!he 
desired,  geneiralizatioa  can  Indeed  be  corried  out.  provided  we  aesviie  that 
the  system  (l)  .has  -cwd  important  properties:  it  is  completely  observable 
and  cosspletely  eestrollable.  Most  of  the  discussion  is  concemed  vlth  the 
case  of  contiauoKW  times  urless  explicitly  pointed  out,  the  tra^ta^-,  of 
the  case  of  discrete  time  is  very  similar.  Of  necessity,  this  section  Is 
rather  technical  and  nay  be  caaitted  at  flr«t  reading. 

f 

In  the  Interests  of  simplifying  the  notation,  ve  aeeujse  that  the  original 
model  .(l^)  of  the  random  process  is  one  in  idxlch  the  cross-covariiOjCQ  aatrlx. 
C(t)  of  m  and  x  ^  aero.  ®ila  does  not  entail  any  loss  of  generality. 

For  let  us  replace  y(t)  by 

f(t)  -  0(t)C(t)R"^(t)H(t)j  (16. i; 

#  ' 

aC't)  V 

G(t)  +  g(t)a‘"^(t)c'(t).  (i6.a) 


Writing 

w(t)  «  K(v(t)|T(t)}  «  c(t)R~\'c)r(t) 


and 

v(t)  «  v(t)  ~  h(^)j 

me  obtain  the  matrix  block  diagram  shomn  In  Fig.  2^  In  which  the  rendoa 
axoitations  x  K  independsEft.  ISbu  effect  of  dependence  bet'i^cen 

T  and  V  in  the  original  model  is  now  represented  by  the  feedforward  term 
g(t)g(t)j"^(t)  In  the  block  dtagram  <jf  the  optimal  filter. 
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Buinoa  frca  uoir  on  S(t)  li  tftktn  to  )r>  IdiKaticAlljr  sero. 

Svvposo  that  £{t|t)  1r  nessia-^lari  2S»a  (HI^)  say  be  wsdttea 
equtfalantly  at 


d£"^(tlt)/dt  .  -»*(t)fi*^(tlt)  -  £‘‘^(t|t)r(t)  +  H'{t)R"\t)K(t) 


“1/ 


-1/ 


i 

Biin  oguatlon  htA  preoiMly  the  aaae  general  fora  as  (lIX^)  but  the  sysbola 

doaiJ^P^tijag  the  xsjvlT  ^OXtBS  cn  the  rlaht  are  ecae'^hat  dlfffarent*  \fe  reg^^d 

(III*)  the  adjoint  cf  (ill  ).  We  can  foraal.ite  this  notion  ao  follows  I 
c  c 

(16.3)  nsrarcrcw,  Ibe  ad.1olnt  of  the  nodal  (l^)  Is  giren  by 


dj^/dt  m  -  r’(t)x  +  5'(t)v(t)^ 

£(t)  -  0*(t)x(t)  +  T(t), 

where  ('■«) 

'  'r 

COT[x(t),  t(t)]  .  3'‘^(t)a(t  -  t), 
cov[;jf(t),  v(t)]  -  H’^(t)6(t  -  t). 


■Riis  definition  is  in  agreeoent  vith  the  usual  terminology  in  dirre.renfciai 
equations,  (in  [U  -  5]  a  somewhat  different  concept  ("duality**)  was  used, 
but  in  the  present  case  the  concept  of  the  fid  joint  is  more  conTeniont.) 


(16.^)  RBMAJRK.  There  la  no  lose  of  genercdlty  In  asBuminjpr  that  the 
matrix  ^(t)  In  (l  )  is  inrertihle.  In  fact,  wo  can  ovon  assume  tb*t  the 
cowarlance  matrix  of  w  is  as  wac  dlscuspod  in  Sect.  7.  Thus  the  adjoint 
system  always  exists. 


It  is  natural  to  introduce  the 


(16. 5)  mnxmm,  a  sywtwa  (I^)  i«  suiiA  to  t»  coaasletoly  ccntroy^b>: 
if  Ita  ftdjoiat  i«  OQHpXetely  olseerrabla. 

m  Tlw  of  (15.7),  coBplBt*  controllebllity  !■  eqpoiTftlent  to  the  follcwins 

(16.6)  Vim.  fta  .getja  (I^)  U  cowplstaljr  oontroll.. 

ahl»  If  «&  oaly  If 

K(V  t)a(t)s(t)g*(t)i*(!r,  t)dt 


la  pohltlTe  definite. 

One  can  of  courae  &lao  define  complete  controllability  directly  [55]* 

She  ayetem  (1^)  la  ccapletely  controllable  if  there  catieta  aooe  forcing 
function  jr(t)  Nhlch  takes  the  aystes  initially  at  reat  (xCt^)  “  O)  to 
any  arbitrary  state  x  In  a  finite  length  of  tine  (x(t)  »  x).  Furthermore, 
it  follows  [33]  that  the  aacrunt  of  "control  energy"  neceaeary  to  accom¬ 

plish  thlP  la  glYen  by 


/  dt  -  ly 

t,,  g-Ht) 


lisji 


Sie  natrlx  ¥,  which  la  the  adjoint  of  the  infonaatlon  matrilx  is  thus 
seen  to  represent  "reciprocal  energy”. 

Just  as  in  the  case  of  Exanplee  (12.7)  and  (l4.11),  it  is  desirable 
to  impose  conditions  which  guarantC'e  that  oboerrablllty  and  C  a/ 1. 2. 
are  egaentl&lly  ^laaffectad  'by  the  choice  cl 


(16.7)  ISFUmau.  A  system  (l^)  Is  imlfomOy  completely  obeeryable 
if  there  exist  fixed  positlTe  constants  a,  a,  p  such  that 


0  <  Off  <  M(t  — o,  t>  at  Bl* 


for  all  t.  She  ayaten  (l^)  is  uniformly  completely  controllable  if 


« 


aav  footnote  on  page  198 


oin 


0  <  at  s  W(t  -  ff,  t)  s  pi* 

for  all  t.  (top  aiJSQ>liolty,  It  is  assuasd  tbat  tbe  aaioe  oonettints 
a,  otf  p  actsvsf  is  both  iss^mlitlas.) 

(16.8)  SMUK.  9ib  trcmsfomatioa  Istroduead  «ct  the  boglnsin^  of  this 
sactlos  doas  not  slTbct  oJtSari^illty  or  centra  >liablllty.  j&sdsad,  if 

X(t,  t)  is  tho  txwyiitlasi  natrix  corresponding  to  (i6,l),  tioeiri 

v(t)  »  g(t)t(t,  T)^(T) 

T 

-  I(t)[l(t,  t)x(T)  +  /  i(t,  T)G(t)g(T)3^(T)g(T)x(T)dT3 

t  ' 

by  (^*^).  Etonoe 
T 

z(^)  /  I(t)t(t,  T)g(T)c(T)R“^(T)3t(T)df  -  i(t)ft(t,  T)x(T)j 

vhloh  shown  that  them  oxlsts  «a  unbiased  estlastor  for  (l6.l)  If  and  enJ^  if 
there  exists  cos  for  Tj  snd  it  la  exsex  that  the  rKrlamosa  at  the 
estimators  axe  ths  j^aae  in  the  tec  cases.  Bmam  tne  aatrlx  M 
in  both  easee.  Pessiaag  to  the  adjoint  ayntea^  ve  reeoh  tba  seas  coLrulw^lcs? 

regsrdlng  the  satrlx  g. 

Enrootfi  to  the  aolxi  theores  ^  cittsS^lish  ths  fwetp:, 

whiah  are  of  interest  In  thaaselres. 

(16.9)  lawd.  Jf  (I^)  is  enifogisljr  ogsayletolr  ohserwto. 

£(^^1^)  Xor.,<aa  t  h  t^  t  n, 

£(tjt)  tS  ir^(t-«,  t)  +  Jl{t-  V,  t)  vbsfi  t  s;  t,  +  tf  *. 

To  prore  thlii,  09  ssaXi  use  of  !^or«a  (15.21)  iSiich  pronridss  a  filtor 
for  unbiased  estlaatlau.  Tkts  filter  is  of  oourse  not  cs^tiieal  in  the  sense 

'  r-.-  —  -r  1.-^  ^  ^ 

See  footnote  on  pam  196, 
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of  Beet.  9,  «nd  thesrofore  parcrldes  m  vopper  1>omyl  for  tbo  TmTiafflice  of  the 
qptiiMLl  filter.  Xov  the  no«t  iMfioriiMixb  foattire  of  rtcblased  eBolxsatlon  is 
the  fact  that  errOBdue  to  the  laitlal  of  x  are  reduced  to  zbvo 

In  a  finite  length  of  tloe.  put  it  niff-rent-ly;  i*.  svsffiBrtH  tu  operate 

on  data  orar  the  tlas  interval  [t'  a.  t]. 

She  comrlance  atribc  of  the  filter  (15.22)  at  arty-  tjta®  t  <#  t^  +  o 
la  therefore  glren  Toy 

,  t  1 

u  (t-ffj,  t)  t  covl  /  dT  ^(t,  '?)g(T)g(‘r)  /  dv  e(T,  v)G(v)w(,v)  1  5 

t-o  (  t-cf 


idjuera  v(t^  t)  ie  tin  traaaltioa  aatrix  of  (15.22).  Mdlng 


r  t 

COf[  /4t«'X(t,  T)l(T)g(T)  /  dv  f(T,  v)Q(v)v(v)j 
t-fr  t 


to  the  preoedlng  expreasion  and  Making  U9e  of  (15.25)  eetahllBhes  the  dosirod 
Ineqanll*^. 


(16.10)  (Ig)  la  uallX)»ly  cganXetely  cqi:trolI^.blc  and  vg:tl.formly 

[S*’^(t-«,  t)  +  t]"^  S(^!t)  >»he«  t  St  -5- 

I«et  ua  ooRsmte  first  an  ea^roaaloa  for  the  rate  of  chacgn  of  the  deter- 
alns.il  of  thfe  ecTarlanoe  antrlx.  Xleoautary  hut  extcnalTe  aiuiipulatlceio  yield 

111  1 

d[dat  E]/dt  -  [2  tr(X  - 

thlch  la  ralld  of  eosrse  onl^  If  det  £  >  0.  We  hare  already  encountered  a 
speolal  eaae  of  thia  foomula  in  the  Mni  (1^.50). 


She  aeooQd  asd  third  texaa  oo  the  rl^t«hand  aide  of  (l6.1l)  are  non- 
negatlT*  heeaaae  they  the  tzacsa  of  nogaagatlrs  definite  e^tricea,  l^cs 


t 

4st  L(t|t)  2!  (*»  £  /  trr5;(?)  „  5{T)g{T)3dT)aat  SCt.jt  ). 

Xov  i;,  E  flyr«  u*UMd  to  1m  a  oontlaaoua  fucetiau  of  t  (mo  8»ct,  li); 
and  so  Is  ^(t)  hy  (5«15)«  ^ooe  ^  li^iS 

dot  £(tit)  >0  for  all  t  >  t^. 

Fro»  tbie  and  fl3-3)  1"^  iolloim  that  if  is  po«lti?«  5sjf Shiite, 

J^“^(t|t)  «xl8t«  ft*  «al  t  >t^  axsd  Is  ths  u&lgua  eol«tlon  of  (m*)  haring 
the  Initial  ralit«  S.’^Ct^l^Q)*  Applylnuj  (16,9)  to  the  adJoSjat  cause,  the  desired 
inequality  fcllom  at  once.  ' 


(continued  on  next  page) 

#  ' 
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(16.12)  I&ttk.  auppoif  tfa6  gyatea  (l^)  is  uaifoCTOy  ec^lgtsl^ 
oontr»T»^^]it  la  noansBstiTB  definlts.  !a»n  .E(t|t)  is 

poBltlTS  dsflnite  fw  all  t  a  +  0. 

'ifs  hST*  «lj*«sdy  saen  in  tfas  ooursa  of  the  proof  of  tlie  preceding  lesea 
th&t  if  E(t|t)  is  ossee  ncnsirsular^  i.t  vill  r«Bi.^  nonaingular  thereafter, 
asnoe  it  is  sufficient  to  tint  5(t^  |tj^)  is  nonsingulnr. 

Let  us  sesune  the  contrary •  Tb»a  there  is  a  fitted  muisero  vector  £ 
such  that 

we  shall  shov  that  actually 


0. 


(16.13) 


»  0 


for 


t  s  t  s  t. 


(16. 14) 


Uhere  T  denotes  the  tzvasition  csatrix  of  the  optimal  filter. 

Let  a(t)  -  Y(t^,  t)E<tlt)f'(t^,  t).  By  <11^.)  and  (III^)  it  fonowB 
that  S(t)  satisfies  the  Integral  equation 

t  , 

S(t)  -  3(tVj)-  /  [S{t)l(t,  t  )H'(t)s‘-^(t)H(t)r(t,  t  )S(t) 
t 

o 


+  t)o(t)g(t)o'(t)i'(t  t)ldt,  (16.1^)) 

^  X 


!2ii:  L-.-U-jiraal  La  deflnltG.  So  Ir? 

,  B  0  by  hypothesis.  Hsnee  W|(t) 

'»hich  proves  (16. 14), 

Instead  of  (l6.13)^  ve  can  write  also 


P^t  )  f>r:  the  hcDvl. 

o' 

rPist  vsirinh  iuttatiealJy  on 


E(t |t)?*(tj^,  t)p  nr  0  when  s  t  t^.  (l6,l6) 


^*1  I. 


LBt 

a(t)-r(Vt)E. 

Differentiating  vlth  respect  to  t  and  using  (H^)  and  ('l6,lH)  ve 

da(t)/dt  «  [<si»(t^,  t;/dt]£, 

-  t  +  g’(t)B"^(*)H(t)5(t!t)]p  j 

Dwa  satisfies  a  differential  equaticn  vhlch  is  the  adjoint  of  (H^)? 

this  equation  has  the  unique  solution 

a(^)  “  t  (16.17) 

i^mch  satisriec  the  initial  ccndltion  «  £«  Subirtltutlng  (16.  l4)  end 

(16.17)  into  (16.1;}),  ue  gst 

K(t)  *  s(t„)  -  K(v  h)  • 

Ely  hypothesis  (16.I5),  wo  hare  then 

-  ^(t„)  -  Ws(v  ‘x)- 

Since  Ji  /  Qj  the  rl|^t-haa&  Is  positive  hy  the  asswed  tnxlfos*  coespleto  controli 
ahlllty  (oa^  (I6.6)).  ttila  contradiction  prowas  that  E(t.  |t^  )  Ir  poaltiw 
definite. 

Ws  are  now  in  a  position  to  prove  tb*  obiof  result  of  the  paper. 
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(l6.:i.8)  MMW  mORDL  an3PBOg«  iasc  1b  mlXoraly  ccctpl-eteOy 

obaaxTal»la  and  uBlforwly  corolatelj  <Hnitrollahl8.  IQaen  tbo  optlaeJ.  filter  1b 
ftgyagytotlffiilly  otatlft. 

(16.19)  RBKAXK.  It  Bboulil  be  akniinciGd  rl(^t  sxsy  that  ”optijLality"  b/ 

no  BsanB  "stability".  Ihit  in  nhyBidsLl  applications,  tlie  unlforw 

asyB(t^otlc  stability  of  the  optical  filter  la  an  Indlapenaable  requirement. 

If  a  systttK  Is  not  unlfoxstly  asymptotically  stable,  then  a  bounded  input  may 
roBiilt  in  an  uabowied  output  I  ].  Hence  (k»r11  biae  errors  etm  ndn  the 
perfOdMoee  of  the  filter.  Perturbations  in  the  values  of  x  would  be 
dlaaataroua  unless  the  filter  is  at  leaat  stable.  *rbr.  search  for  conditions 
undbr  which  "aptlnality"  lapUes  various  fornus  of  "  stability"  is  the  central 

tl  ' 

problaa  of  filtering  theory.  In  the  classical  Wiener  approach,  this  problem 
Is  ecBqpletaly  Ignored,  but  it  turns  out  (see  belov)  that  the  classical  asBump- 
tlooB  guarentee  stability  anyway. 

(16.20)  KXMIFUBS.  Ihe 'candltlODS  of  the  theorem  are  clearly  satisfied 

in  case  of  Siaagples  (l2.l).  Case  (1t)j  (12,7);  ^th  >  0;  (lU.ll). 

In  these  aacsstples  ve  ware  able  to  show  unlfom  asyaiptot  lc  stability  of  the 
optimal  filter  by  d3.rect  ■ethods. 


(16.21)  CKJUraESKIMCPI^.  Shat  happens  If  we  do  not  hare  complete 
controllsblHIgri  In  Sxasqple  (iC.l),  Case  (ll),  with  ^  ***  Dp’tjJTiH.*. 

filter  Is  stable  but  not  ahysiptotloally  stable.  On 

the  Other  hand,  la  Case  (ill)  of  tha  same  axsaplf;  the  optimal  filter  Is  iiayapto- 
tloally  stable.  Similar  oamaants  apply  to  SsobiqiIo  (ib.l).  Another  llli^erratlcm 
of  th»  theorem  is  prorided  by  Case  H-A-l  of  ttsamplc  (lii.20).  Every  steady -state 
optlaal  filter  corresponding  to  the  matrices  on  page  ITS  is  xTsstable;  the  cigen- 
vralues  of  P  -  Iffl  are  +10  In  each  osso. 
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%)  be  the  traneltion  netrix 


of  the  optiaal  filter  (H^).  Tbaaa. 

t)  .  X'ir,  t) 

la  the  trenalticn  natrlx  of  the  anoint  of  (H^/j 

dx/dt  «  --  Li;*(t)  -  K*(t)R“^-{t)5(t)E(tlt)]x.  (16.^) 

Ve  shall  p*c-:re  that,  fw.  .ill  i,  ther^a  are  pr^aitive  conatauta  c^,  such 
that 

f 

‘•'O* 

ll^t,  t  +  o)i|  c^e  ^  •  (16.23) 

Bxla  implies 

H3 

Ilf  (t  +  0,  t)!!  -  lift  +  tf.  t)l|  iQ^e  ^  ^ 

end  thiia  he  uniforta  asymptotic  stability  of  (H^). 

To  eatabllah  (16.25),  'f*  Irtroduce  the  lyapunov  fvxnctica  [l4] 

v(x,  t)  .  |)xl|^^  . 

By  leiBBAS  (16.9),  (16.10),  (16.12)  ve  knov  that  V(xj  t)/||x||*^  ie  uniformly 

bounded  froja  above  and  below,  at  least  for  t  i  t,  »  t  +  2fji 

1  o 

(a  +  p"^)  11x11^  s  V(x,  t)  3  (a"^  +  6)!!x|!^.  (15- 2i^) 
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Tf5:ieli  viV.  proT«  (16.25),  in  ot  the  nAl-taoun  theorem  of  lorapunov  [Xh], 

She  problem  !•  to  find  a  lover  hoiaad  for  the  Integral  of  the  rl^ht- 
hand  aide  of  (16.25). 

Let 

e 

il(-r)  -i  S”^(t)i(>t)£(T)/v(T,  t  d)s(t'  -!-  a). 

by  (16.22)  and  (V.p) 

aCt,  t  +  c)§(t  +  ®)  ••  i*(‘t  •••  o»  f  +  o) 

t-+<T 

/  ®*(vj  T)H’(v)u(v)dV. 

T 

t+0  n  ~  r, 

I  Il!l(v)ll5(y)4v  -  «^!!x(t  +  c)lj^j 

}|0*(v)a1[v,  t  +  tr)x(t  +  G)\f 

V 

Tj  depend  on  t  +  or  and  s(t  +  o).  'Jence,  vrlting 
▼(iC't  +  0),  t  +  o)  -  V(x(t),  t)  *■  ft^l|x(t  +  o)ii^. 


Let 


and 


vbere  c  and 
6^  - 


Further 


p..  .  .0  2 

Tl  iix(t  +  Cf)|j~  %  /  |iO’(T)*'(t  +  o,  T)(x(t  -i-  c)  +  /  'f’Cv,  t+o)H’(v)u(v)dv]|i  ,  dt 

t--  "  t"  "  H  (t) 

(16.27) 

How  if  tSTSti  +  tf 

t+0  .0 

II  /  0’(v,  t  +  o)H»(v)u(v)dy|r 

t 

S'  [  /  l!d*(v,  t  +  or)g'(v)5^(v)||*||u(v)|jj^^^^dvl^ 
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Sly  Stolnmrz's  ln«au&llty^ 

S  €^[tr  lf(t,  t  +  o)]||x(t  +  c)!!^p 

Expanding  tbe  Itrtagrtod  In  (l6.<!7)  and  using  agai-i  Schuftsfit?  e  InequoliTv;,  wb 
see  that 

T}^!|aE(t  +  0)11^  a  I!x(t  +  ^  ^ 

-Sfitr  W(t,  t  +  0)][tr  ^(t,  t  +  ff)]^l(x(t  +  (y)i!% 


By  unlfora  Cf^iiyslote  obearrablllty  andfCootroJJLahiiity, 


r  -  +  n  i  o  -  2  <V  '  ^ 


-  2  g  +  f ' 


Moreover,  wb  have  crivlally  also  that 


2  2 


Combining  the  two  preceding  inequalities,  we  iind  that 


6^  ..flS-  >  0  . 


Hence 


v(x(t  +.0),  t  +  0)-  y(a(t),  t)  i  V(x(t  +or),  t  +  e) 


fhls  estahllahae  (16.26),  and  ccapletes  the  jpnxd  of  tbe  theory. 

Our  g*ltt  thaaraa  has  sn  lasMdlate  and  lii^pcrtsnt 

(16.26)  nCKMC.  aa8»..tto..,|g|eg  (l^) 

teiaii,r  u.  mSi 

sCT.ja:  la?  igMtaw  <tf  tti  orodasa  ^^^mjLmmss^SssL 

daflalta  valw  at  t  ->  t^,  thaia 

+  ajt  +  0)  -  s^^(t  elt  ^  0)1  a 
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'Suit  iB,  the  effect  of  the  Initial  state  ?(t^)  is  gradually 
"forgotten*^  eus  t  -too.  Bila  la  Important  In  practical  eppl-leatlons,  l^cau&e 
the  Talue  of  £(t^)  nay  not  be  accurately  knovn. 

Kunerlcal  Integration  of  the  yarlance  equation  is  facilitated  because 
the  effect  of  round •^^ff  errors  will  not  be  cumulati-vo . 

Proof.  Let 

Ni(t)  -  -  Z^^^tlt) 

It  is  busily  rerlfled  that  5E(t)  obeys  the  differential  equation 

-  5^*^(t)H*(t)R’’^{t)H(t)]az(t) 

+  6&(t)[F(t)  -  S^^^{t)H'(t)R“^(t)H(t)]' 

Trodi  thlS|  it  foUovs  easily  that 

Ofi(t)  -  t^)hE{t^)f^^^*(t,  t^)  hh.^) 

uhich  is  the  sinaQog  of  (l^,lB)j  rasp.  >»  the  transition  matrix 

of  the  optlnal  filter  corresponding  to  and. 

Daklng  nones  In  (l6.29)  and  Inroking  (l6.24)  prores  the  Uieotxitn. 

Consider  aow  the  solution  of  the  rarlance  equation  conrsapondlng  to 
"  Of  uhich  ve  denote  by  E(t>  t  ).  Under  the  hypollseaee  of  ( 16.26), 
the  iiiilt 

£(tj  SI,  t^)  -  |(t)  (16.50) 

t  -frvjoe 
O 

exists  for  all  t.  To  prove  this,  it  is  only  necessary  to  note  that 
£(t|  Of  t^)  is  noodecreaslng  vlth  t^,  l.e., 

£(t|  g,  t^)  h  £(\|  fi,  \)  (16.51) 
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vhenoYer  *  t^.  awn  (I6.3O)  feliova  iiy  atandJiupd  ccnTorgencs  argunente 
slncft  (16.9)  £  la  unlfo?ttly  'boundod  frota  abore. 

To  prov«  (l6.5l)>  let  and  bo  the  traneition  matrix  and 

K'°^  and  be  the  gain  of  the  ortimal  filter  corresponding  to 

5(ti  0,  tj  -  cot{  /  l^^'(t,  f)[Q(T>j,(T)  -  K^^nt)l(T)]d'r}  . 

t 

o 

!1^  optimality., 

t 

S  C0T(  /  X°(t,  T)[a(T)ir(T)  -  K^°^  l  )v;,(l  )]dt), 

o 

which  vaa  to  be  proved. 

Hence  vo  have,  as  an  iMnediate  corollary  of  (16.26), 

(16, 52;  rQi]h3Ri&4.  Suppoee  that  the  system  (l^)  is  onlformly  cowpletely 
observable  and  uniformly  coapletely  controllable,  then  every  solutlcai  of  the 

variance  vhich  has,  a  nonnegatlve-^-cfinite  value  at  t  *  t.  conveii^es  uniformly 

o 

E(t)  defined  by  (16.50). 

In  view  of  this  theorem,  wc  caLl  E(t)  the  moyina;  equlllbrliaa  state 
of  the  variance  equation.  In  the  case  of  constant  system  the  solution 

of  the  variance  equation  dependa  only  oii  the  dirrej'rM-.  <-.1  t  -  t^.  Scace 
^(t)  ^  ^  5=  C'^St. 

(16.55)  IHEOREW.  Suppose  the  random  process  x(t)  la  generated  by  a 
constant  system  (l^)i  i.e.,  f,  G,  H,  Q,  R  axe  constMits.  Suppose  fuidlier 
that  (l^)  is  completely  ohaervable  aixi  completely  controllable.  Ihen  every 
solution  of  the  variance  equation  vhich  has  a  ponnegattve  definite  initial 
value  tends  uniformly  to  a  constant  matrix  L  in  the  limit  t  *=  00.  Ihls 
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watrlac  H  tto  imiaw  BOftltJlTO  dtflalte  •<mlllbrluiii  at^ta  of  the  rarlgace 
toiifttlon.  ii*.#  It  In  thiB  mlgUB  PoeltlTe  defUBl-fcg  solution  of  the  ayetcm 
of  nimiXimsmooM  <nwdr»tlc  tlgil)re>lc  equfttiona. 

d5(t)/dir  -  g 

rK>or.  Tao  lirBi  paxt  of  the  thoorem  follows  at  once  from  (l6. 52)^  L 
Is  positive  ‘.'sufinite  "by  (l6,12).  It  is  unique^  becaune  if  (lIX  '  hnr. 
than  one  conet^uat  solution,  (l6.28)  is  contradicted. 

(16.34}  EXAMPUBS  AND  CODlfSEiaEXANPXXS.  Ccnsldkir  Example  (l4.20).  We 
always  baTS  complete  obsarwabillty.  If  det  g  >  0,  or  det  Q  -  0 

/  t^,  then  we  also  have  congplete  controllability,  and  Hieoreo  (lij.  !i8)  , 
which  was  proved  by  direct  methods,  shows  that  E  Is  unique  funl  pooitive 
definite.  On  the  other  hand.  If  det  g  -  0  and  then  Tiicorcm  (l4.48) 

shows  that  ^  thotxgli  possibly  unique,  will  always  be  singular.  Hence  the 
condition  of  complete  controllability  cannot  be  dropped  fran  ( 16.55)* 

/ 

In  Bxaii5)le  (l4. 50)  we  have  conqjlete  obcervabllity  and  cmplete  controll- 
abilityj  the  equilibrium  state  L  given  by  (l4.5l)  is  Indeed  pooitive  defiiiXte. 

On  the  other  hand,  if  complete  observability  la  aeatroyed  by  oettitig  0, 

then  there  doea  not  even  exist  an  equlllbrivaa  state  ('unloBG  =.  0  which 

ms«uiB  that  the  second-order  problem  la  degenerated  Into  a  first-order  one. ) 

Hence  the  condition  of  complete  observability  cannot  be  dropped  from  (l6. 55). 

In  Example  (l4,52),  we  have  complete  observability  but  not  comrjlete 
controllability  since  g  «  0.  Indeed,  we  see  from  (ll:.5'r)  that  all  aol^itiom! 
cf  tb^  equation  approach  2  «  0  as  t  -»  oo.  Thus  In  the  absence 

of  complete  controllability  we  connot  guarantee  tliat  5^2* 

It  lo  not  clear  a  priori  whether  or  not  the  ussumptlono  or  the  clasaicol 
Wlener  nT-nblem  imply  complete  observability  aiid  coaplcto  controllability.  Ln 
fact,  the  answer  lo  yes. 

For  if  ^I^)1b  not  completely  observable,  we  can  introduce  apoclol 
coordinates  so  that  the  defining  equations  assume  the  form  [5?] 


t(t)  -  l(^)» 

ts) 

In  other  vords,  eoua  of  the  state  variabXas  (namely  the  cosapac^-^tiu  of  x'  ') 
do  not  affect  z(t)  -»  hence  they  may  he  l^iCiXd  in  tho  stato  O^  *vuu 
filtering  ptoblem.  , 

A  Blsiiiar  decomposition  holds  If  (l^)  is  not  comqpletely  controllahle^ 

dx(2>/it  - 

(2) 

In  other  words,  no  random  excitation  acts  on  the  vector  x'  The  aseuBp- 

C2) 

tl'V'  of  statlonarlty  in  the  Wiener  problem  inplics  that  we  igjat  set  i'  '  «  0. 

Die  statanent  of  the  classical  Wiener  problem  does  not  explicitly 
Involve  X.  Therefore  in  setting  vp  the  presentation  (l  ),  there  is  no  lose 
of  generality  in  asauming  that  x'^'  ic  absent  from  the  j^codlng  equations. 
This  proves 

(,16. 55)  IBBSQBt.  Tjtie  oliMisloaX  (artatlopary)  Wiener  prohl^  coriNS spends 
iSLa-aSfiSi  (Ij,)  is  owl^tely  obmgm^Xe  and  coeipXet^ly  cc^trollcbl^. 

A  sinpla  example  Sj?  provided  by  the  foULoviog  specia.1  ctaso  of 
(14.20).  Lot  dot  g  -  0  but  ^11  "  ^22  ****^««  ^ 

change  of  coordinates 

«■  &  and  (S  M 

where 
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•  ■■■■MMaaapmMaw  m 

T  - 

* 

the  aquations  of  (l^)  aaauBM  the  fora 

*1  “ 

*2  “  ^11^2* 

Since  f^  <  0,  the  second  equation  nay  be  dloregardisd  and  the  Wiener  pro- 
blen  reduces  tc  a  flrat-order  one. 

It  should  he  noted,  honwrer,  that  in  numerous  eqjplicatlomj  («ecj,  c.g. , 
Xraagple  (1^.52))  the  Wiener  foxmLls.tlon  is  not  sufficiently  general,  and  in 
such  cases  q\iestlone  of  ohserrmhillty  and  controllability  may  present  some 
noDtrlTlal  prohleac. 

In  the  elaaslcal  theory  of  the  Wiener  problem,  the  procesB  of  oolutlon 
ixrroltes  the  spectral  factorisation  of  fourler  transforms  Into  two  component 
which  are  analytic  in  the  upper  ree.  lower  halTCO  of  the  complex  plane. 
Intuitively,  this  procedure  is  related  to  the  fact  that  the  eigcnraluoH  ot 
the  matrljc  defining  the  haalltonlan  system  (V^)  (see  Sect.  I5)  occur  in 
paired  If  X  i<»  an  aicenrolue,  so  is  -X. 

now  show  that  under  the  hypotheses  of  the  main  theoren,  this  rcoult 
Cia  be  ^generalized  tn.  nonconst&nt  tjystomB  a»  well. 


TH801RIW.  f?\!rrofe  the  sviftew*  (l^} 

bhserrahle  and  unifowaly  casoletely  oontroHahle.  Then- there  exists  a 


•uch  that  thfc  hial.ltoni»n  attuKtelOMi  (t^)  Mitaift  tha  lUnewiia  fom 


4|/4t  «  -  I*(t)t  , 

(yi  ) 

da^at  -  f  (t)ir.  ® 

vhTe  f  (t)  i»  thft  lnfinlt»alM>3.  tgauMltioia  wstrle  o?  c^tiaal  flltg-r 
corresponding  to  SCt). 

j^c-yovr.  Tft)  and  its  inverse  sy  unifOraaly  boundad  for  all  t. 

IijjQf.  tjat  l(^>  t)  (t  fl3ced^  %  ysriable)  ’.?)«.<  ^iraasltion 
Hstrlx  o«  'tbs  opttnsl  niter^  corT*espot)di.&s  ''co 

Kt)  -  F(t)  -  St)g«(t)»"^(t)H(t). 

The  notions  of  the  optinsl  filter  nay  be  denoted  by 

±\^)  * 


The  scalar  function 


V(x(t),  t)  -  |x(T)f 

5"  (f) 

tends  to  0  vlth  t  -»  oo  in  rlev  of  the  main  theoren  and  of  the  lensaas 
preceding  It,  Differentiating  vith  respect  to  t,  wc  obtain  at  Integral 
expression  for  Vt 

v(i(t),  t)  -  y(i(t),  t)  -  A||H(r)x(T)S^  *  lig'(T}r^fT|TM-v)j’^ 

t  "  "  K‘^(t)  ■”  '■ 

This  proYcs  that 

S(t)  -  lln  /^|a{')x(‘r)ll^  1  dt 
T-»oo  t  R-"(t) 

exists.  Differentiating  with  respect  to  t,'  we  see  that  3  is  a  oointlon 
of  the  differential  equation 


(16.57) 


dg/dt .  -  f*(t)§  -  g(t)  +  r (t)3“^(t)i(t). 


Hov  WB  define  the  tranaforaatlon  T  by 


T(t) 


[  I 

i(t) 


‘  &(t)  ' 

l  +  £(t)3(t) 


(16.38) 


IirtUlsinK  (l6.37-fi),  ts*  eee  that  the  nev  Tarlahlee  (jj^  tt)  satisfy  the 

f 

oanoeloal  differential  equations 


d{/dt 

r  --1 

‘-F*  5*91  ” 

r 

-  1  -IT  +  S 

) 

dH/dt 

>*  IM 

292'  t 

. 

T 

m 

-  F*  0 

' 

1 - 

1 _ 

1 

ia<i 

0 

_ 1 

Finally,  ve  note  that 

-  §(t)  *  <  H(»-'  “  o,  ^•)  +  -  O;  t), 


CO  that  lia(t)|{  Is  uniformly  boicided.  The  ptoof  Is  coaqilete. 


With  the  aid  of  (16, 3l8l)  ve  cen  vrlte  the  oatrix  ©  occurring  in  (15.II) 
in  the  canonical  form 


(16-39-^ 


e(t,  t) 


’  I  5(^)  ■ 

■r(v  2 

"i  +  §('t)i(t)  •^9(t)  ■ 

|(t)  i^,.|(t)S(t) 

0  F(t,  t^) 

. 

”  £(t)  I 

If  Sb  5^  St>  B  con8t«nt,  than  so  is  g  and  and  the 
detannlnation  of  3  10  reduced  to  the  elauentaxT'  prc^leei  of  solving  the  aet 
of  linear  equations  In  the  eoefflcleste  of  S  obtained  by  setting  the 
hand  aide  of  S  equal  to  eero.  Moreorer,  In  this  special  case, 

l(t,  t^)  =  cxp[(t  -  t^)?], 


vhich  cen  be  explicitly  conq;>uted  as  a  aatrlx  vho.  s-c  i^Xswents  consist  of  suns 

>.i(t  - 1„) 

of  exponential  e  ,  where  are  the  elgeirralues  of  T.  Ttils 

shows  that  the  solution  cf  the  cXaesl'cal  Vlener  probleia  luider  the  auurkovian 
assunpticn  contains  In  It  also  the  solution  of  the  problem  with  finite 
observation  interral  (t^  -  oo).  Dius  wo  hacre: 


(i6.ito)  mtom. 


of  (  12.WS  ??)  aay  solutica  of  the 


variance  equation  can  bo  erpreesed  in  closed  fora  by  the  followin 


(i)  yia<i  I  ^ 

(m^)  equal  to  aero 
(11)  find  a 
(ill)  detegain 


> ft -hand  side  _of  (16.57)  SS 
of  y  (which  will  alwwrv 


tn  negEstlve 


(iv)  r4gpAest  e*p[(t  -  t^)|]  ts!3 
(▼)  eexyute  fi(t,  t^)  sx;  (l6.59)l 
utilize  (13.12). 


of  y  I 


(l6.!^l)  ECAMTIJ;.  As  an  illustration  of  this  theosna,  m  ^ 

expression  for  d  In  Sxaaple  (li^.l).  Ilhe  coastanta  0..  asu 

U.  JLO. 

given  by  (ll».3)  and  (lA.7)|  -m  am  that  s^  oibays  tha  mq^Xna 
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or 


'll 


>  0 


u'u  *  Wu 


Bniibirtlttttiiig  thiese  T»lvies  of  Into  (l6.S)>  vo  verify  the 

prorlcosaior  glren  fcamklai  (lli^,lU). 

It  may  "be  Added  that  Ssanple  (!*»■«  l)  wee  c^'^i5ldo^ed  prorlousl/  by 
SKdnbrot  [8,  SnMple  2],  by  his  apscial  ndthod  of  aolvlng  th*  Wlerer-feopr 

f 

intesnd.  expiation.  With  the  new  Method,  the  solution  of  the  integral  equa¬ 
tion  la  ttroided  and  tho  algehralc  nature  of  the  problem  (which  is  the 
ejqplaiiaitlon  for  the  poaalblllty  of  obtaining  rosulta  in  closed  form!)  is 

clearly  erident. 


(16.  h2)  mmssL,  AH  prvriOua  cOu»lu£ii.'<»tioBi$  can  be  oasxly  cau-ied 
*  anrer,  Kitatis  anitaudla,  to  the  caae  of  discrete  tlae.  There  Is  only  one 
point  aliieh  xequlzea  oautlcn.  Xu  writing  down  the  discrete  analog  of 
(l6*io);  ute  al^t  pusslad  by  the  appearance  of  certain  additional  tonns. 
Bat  these  tem  all  cancel,  by  Tlrtue  of  the  peoudo-luTeree  leiwca  {A.k). 


Acpmdix  A 


In  aatrlx  CiklevdAtlosis  tl»re  1b  a  frBq:ua«itay  r*e\urring  dijn.*leulty  due 
to  the  feot  thet  the  luTesrse  of  »  wtrlsr  doss  set  sltsys  exist.  !iy?  prore 
the  existenoe  of  a  given  mtrix  is  often  owherBcne  and  difficult.  Itore- 
ermr.  In  Bigaiy  oMea  aoltuloni  of  a  set  of  linear  equations  exist  even 
the  luTerse  of  the  aatrix  defining  these  equetiscs  does  not. 

To  alleriata  e^  (thcmid’t  not  all)  such  <iiff5.eultlas.  It  has  haen 
found  eosiTanient  to  Introduoe  the  notion  «f  (ho  ,jiO*^<^alled  psaudo»-iJ3versa 
of  a  vatrlx.  Roui^Uly  speaking,  a  pao’jdC'^iBverse  must  possess  tvo  prcn^rtleu 
to  he  useful!  (l)  it  anst  alwjre  exist)  (ii)  ehen  used  in  place  of  the 
lnrerso  (shich  neor  not  exist  it  should  give  the  correct  ansver  to  such 
questions  as  solutions  of  equatlsns.  In  general,  the  pseudo-lnyerse  Is 
not  unique)  this  glres  rise  to  certain  ccnpllcetlons. 

Th»  nsterial  ehich  fellovs  provides  the  naln  facts  needed  in  this 
paper.  ?or  fvsther  details,  consult  Penrose  [5^-tOj  tmd  K^lrean  [%1. 

A  lestrlx  A^  is  called  a  paeuda-inverse  of  a  rectangular  (net  neces- 
srrily  square)  Matrix  A  if  it  satisfies  the  follovlng  relatlcn 

(1)  ^^A  -  A  • 

If  A  has  an  Izrrersa,  it  is  equal  to  A^.  Pear  then  (l)  Ijupllec 
AA^  ■  I  and  A^A  ■  I)  as  is  sell  knovn  p,  62],  these  tw  relatloas 
lnq?ly  that  A  ^  «  A^.  Proa  (i)  m  see  also  that  (A*)^  -  (A’)*. 

It  is  easy  to  prove  that  a  pseudo- inverse  satisfying  (l)  ejsdsU  for 
any  rectangular  matrix  ^  ¥e  ahoy  this  first  for  a  noouegatlve  derinite 
oatrlx  P.  It  is  veil  knovn  in  nunarioal  analysis  that  every 
nennegatlve  definite  Matrix  can  he  transfomsd  to  a  diagonal  form 

T’Pf  -  E,  (A.1) 

vhere  T  is  noneinguler  and  the  Matrix  f  is  diagonal,  having  only  zeroos 

~  2  ~ 
or  owes  on  the  diagonal.  Thus  E  =  K,  Iben 


satisfies  (l).  His  ean  nov  defljsa  a  pMuao-ixrrsrsa  of  a&  axtdtra 


or  by 


-  (A*A)V*, 

« 

-  A»(AA*)^ 


A.  5 


file  pMu&c-imremMS  occurring  on  the  rlg^t  ^>re  defined  by  (A.  2). 

To  show  that  (A.>i*%)  actually  satisfy  (l),  we  need  a  sinple  leuna, 
which  is  Ihe  chief  tool  In  appllcatlans  of  the  pseudo-lmrerse  as  for  as  this 
paper  Is  concerned: 

(A.4)  PaWIDO-ISVIRa  UBOM.  I^|t  (^),  l  -  I,  H,  be  arblti-ax-^- 


for  all  1  «  1,  ,,,,  K.  Btadlarly.  let  B(t)  be  on  arbitrary  tn  x  n  taatrix 
whose  elsaents  aw  continuous  functions  of  t  in  the  interval  [0,  T].  Ihen 


T  ♦  T 

B(t)  -  B(t)i  /  B’‘(T)B(T)dTl^[  /  B'(T)B(T)dT]  r,  0 
“  o“  “■  o"  ~ 


for  aH  t  €  [0,  T]. 

Proof,  let  ^  respectlYsly  0(t)  denote  the  matrlcea  on  the  laff.- 
hand  sides  of  the  preceding  equations.  Using  (l),  we  find  tJaat 


H 

B  C'C.  -  0 
1«1  -i-i  “ 


2%) 


T 

/  D‘(t)D(t)dt  -  g. 
o’” 


Bsnott  for  any  ^ 


and 


Coasequently 


£  -  0 


/  lla(t)xji^4t  -  0. 


ii^*S  -  m^m  - « 


for  all  ^  #iieh  Ixpllaa 

«  g  (i  »  *.  •••f  *)  ^  ^  (0  <  t  fi  ?); 


tba  Xaaaa  la  prorad. 

Mk.tltlrtiW  (A.5.)  l«to  (1),  «!»« 

K*  «1«B  >y  (A.3.)  1*  .  (*-»  !• 

*.><■,  troQOM*.  Btao*  (A.5<i-*>  •»  1“  »m»1  <*•  w  «»  laat 

tha  paoado-iuTarao  ia  not  vilqna* 

Ite  coBsmtJxg  tha  pntyto-Unraraa  of  a  aatarSx  i&lah  la  srjt  a«»r«,  ®j* 
wold  naturally  choocs  that  ona  of  foncslac  (A*5a-h5  to  ahioli 
natrlx  A»A  or  M*  la  -fcltor.  »or  to«tiaa»,  tto  pa«al4i-lwr»»  <tf  « 
ractor  (l-neola*®  natrlx)  la  gtono  If 


=  (x’x)  'x*  “  x'/||xl|‘‘  . 

For  Many  purposaa,  the  lack  of  uniqueneeB  of  the  pseudo- inverse  is  incon- 
TenleEt,  2Mb  difficulty  can  be  cverecs*  by  adjoining  tc  (l)  Vuo  follow- liig 

further  axlowt 

(11)  «'A^ 

(Hi)  (m‘^)'"M‘^/ 

(Iy) 

f 

These  aadoas  vere  introduced  by  Perufose  [59],  who  proved  the  following 

(A.^)  mxam  (Penrose).  For  erory  rectangular  Matrix  A,  there  ejJ.ot9 
one  and  only  one  natrlx  A^  satisfying  Bljaultaaeously  (l-iv). 

^  To  avoid  confusion;  ve  shall  call  the  (unique)  aatrix  given  by  Penrose's 
Theortte  the  aenerallted  inverse  of  A  and  designate  it  by  . 

Panroee  (see  (i®],  with  slight  modifications)  has  proved  also  the  follow¬ 
ing  livortant  property  of  the  generalised  Ixxverset 

(a.0)  THBOm  (Panrose).  Consider  t.he  equation  JU  «»  b.  x°-  A*b, 

«d  X  /  X®.  Ttetnt 

11^  *'  ^ 

(^)  ^  -  "bfi  -  |to°  -  58  1*1  >  |*°li- 

Xn  other  wordS;  x''  •  jjib  glres  the  solutiou  of  to  "  b  if  one  exlsty 
and  the  best  agpiroslmatlom  to  a  solution  ahera  aoass  eaLir^.e,  Thu*,  wj  th 
we  can  call  x®  the  best  amaaroxlmate  aolutloai  of  ^x  *  b. 
let  deawte  the  foHovlag  nesm  ef  tha  ^rlx  ^ 

|4(^  -  trace  4»A  -  aj[j  .  (a.?) 


OHO 


(a. 8)  COROUAKr.  I«t  ^  f  A*  bi  %  wrjdo-lgytrtfw  of  AJ  aay 

mArix  vhleh  eatlSAlss  (iU  Tbaa  >  jA*j. 

In  other  vorde,  the  senerellted  Ixrrerse  is  (in  the  oenee  of 

the  no»  (A*7))  UQr  other  peeudo- inverse. 

To  prove  (A.8),  it  suffieee  to  note  that  the  »trlx  wimtioa 


ASA  "  A 


*ey  he  interprete*!  a  vector  equation  in  the  eJLeaente  of  tiv;  natrix  X.  By 
+  ~ 
hypothesis.  A'  is  a  solution  of  this  equation,  so  that  case  (b)  of  (A.6)  1b 

applicable.  Q.E.D.  ' 

To  Illustrate  this  result,  consider  the  matrix 


A  • 


1  1 
1  1 
1  1 


Two  pseudo- inverses  are  given  by 


“l/3  0  O" 

f- 

o 

_ \ 

0  V5  0 

and  Ap  -  !  1/6  l/6  0 

o 

o 

& 

— 1 
*fN 

o 
_ ( 

and  the  generalized  Inverse  in  given  by 


lAii 


1/9 

1/9' 

1/9 

1/9 

1/9 

f 

1/9 

l/9_ 

-  1/5 

>  1^! 

-  2/9  >  |A*1  -  1/9 

80  that 


Sm  geiMZ«lis«A  inv*ni«  is  «Tl<l»stjly  tmlquftlsr  determinBd  by  (l)  Aztd  the 

remjiraBfist  thst  it  is  t.hs  $««udo-lBv«xiie. 

FLsAlIy^  let  US  MBtloii^the  i^nenlised  Icreree  cen  be  detexvlned  by  • 

sBtthod  elidliir  to  (A.1)  thrwaglSi  (A.3)<i  Tble  Is  done  [UcH  by  Iterating. tvlce 

ti«i  algorttlMi  which  detdrsiaoi  T  eatlefSTlng  -<  £. 

« 

Let  X  be  an  n-dlnene local  raadoa  vector  with  neac  ”*  eCx) 
aace  natrix  ^  -  *(»*)•  It  la  c^Jitaaary  to  say  p.  17]  that  x 
If  Its  pcrobablllty  density  m'ctlon  Is 


and  covarl- 
ja  ssuBslaa 


(B.I) 


This  definition  of  a  gaussian  randea  vector  does  not  appiy^  hcveycr,.  vhen 
r,  is  slacular.  Tn  this  case  the  vaLluen  *  by  »  eorri’ined  vith 

probability  1  to  a  hyperplane  of  dljnenslon  less  than  n,  end  on«  cannot  e.rprcfl8 

thlB  fact  by  a  foraula  such  as  (b.1).  Consequently  If  £  la  s*lnguinr,  the 
proibsblllty  distribution  of  x  Is  defined  by  first  introducing  a  linear  trana- 
^ocnsstlon  *  "  ^  tt  (where  jr  is  a  n-rector,  a  being  the  rank  of  £),  such 

that  the  covariance  aatrlx  of  jr  is  nonsingulnr  [4M,  p.  26]  and  fiCy]  »  0. 

Then  the  prohablllty  doaslty  function  of  y  can  be  expressed  by  a  fortsula  aiie- 
logous  to  ( B.1). 

These  owlcwaj*d  difficulties  caused  hy  the  singuiarlty  of  £  can  be  avoided 
if  one  chooses  as  the  basic  deflnltlou  of  gBUSslanness  the  characteristic  faucUon: 


(B.?) 


S^(i{)  “  Etexp  is’^ 


exp  lls'ji 


+ 


In  this  definition  the  Inverse  of  £  is  not  req.ulred. 

Since  the  distribution  of  e  s&aasiftn  rondcei  rector  is  xmiquoly  d«tera.ined 
by  its  aean  and  coverlence  aetrlXj  it  is  deslrsble  to  cel.culate  as  mucii  as 
possible  directly  vlth  and  ^  .  For  these  juxposes^  (b.2)  Is  batter  ^ulted 
than  (B«1). 

Slsilarly^  a  pair  of  gausslaa  randcn  rectors  Is  defined  by  their 

Joint  ebaracterlstlc  fuxwtiou: 


«*!>  (B. 


vbez« 


%  -  1,  J  -  2- 

It  fcUoim  fica*  (B.3)  that  and  Xg  are  Indaijwwiident  if  and  only  If 

«  S.  aiailarly.  If  x^  agd  3^  are  gaamslan,  then  og^  +  pofg  +  c  le  nJLso 

Ito  BCsr  proceed  to  daidre  explicit  «xprei»ion»  tor  the  coadltlonal 
and  conditional  canrariaiiou  of  a  pair  of  gausslan  raadm  Teetore.  To  do  this 
elesantly^  -vs  naka  usa  of  a  meant  otaaerratioa  of  BalakrlsboHu  [^1  vhlch  ra- 
Istes  these  ^uaBtltias  to  the  Joint  oharaetarlstlc  function.  In  a.  sll4ghtly 
aodlflad  fom,  this  result  isi 


The  copditlQpal  exmetetion  of  cLren  Xg  is  a  linaax  fuaction  of  3^ 


(He  4) 


(i)  MJS4  (ii)  sak-i? 


(b.5) 


vhflx«  ^  denotee  the  rector  vlth  ccn^ionexilB 

Proofi  Since  every  Bcwent  of  a  gausslan  dietrltoution  ie  flnitej  it 
foUoKTS  tgfg,  p.  67  and  89]  that  we  say  interchange  differentiation  with  respect 
to  is  with  the  expected- value  operation*  Than 


Tilting  the  expectation  fint  with  respect  to  the  conditional  probability  distri¬ 
bution  of  x^  given  3^,  and  then  tlth  respect  to  get 

"  K[x^|xgl). 

,  Utilising  (B.4), 

Interchanging  I  and  d/dn^i 

■  '“12  *  s  V^'’ 
which  porcves  part  (ii). 

To  prove  part  (i),  we  proceed  as  before.  Interchanging  S  and 
vhieh  leads  to  the  relation 

isJLsi- 

Bg  -  l(xj^jxg;]e  .  0  , 

valid  for  oviry  Sg,  This  lapllas  (B*4)  by  the  unlqueneee  of  the  fourior- 

atleltjes  traMfens* 
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We  nov  state  the  oaln  result  of  this  seatlou: 

(B.6)  THiaMW*  The  ettodltlonal  eapeetation  of  x,  glTeu  Xg  Is  a 
gmselea  runftca  fctqr  glYin  )Br  £12^2^^  ~ 

Thlfl  is  the  foanula  found  In  textbooks  p,  28],  except  that  hers  the 
inrerse  ie  replaced  by  a  psaudo-isrrerae.  Xote  that  if  ^  aUnoet  surely* 

!•••*  ••  0,  then  (B.6)  Is  correct  since  0^  ’’  Q. 

To  prove  (B.6},  ve  utilise  Strelghtfonrard  differentiation  leads 

to  the  condition 


Since  this  eust  hold  identically  in  s^,  we  see  that  K  jmiat  eatlsfy  th?  equa¬ 
tion 


£i2  “  ^22* 


f  V  'y\ 
\  !><■  ( ; 


We  now  show  that  this  equation  always  has  n  solution,  which  con  h-A  expiressed  m 


K  ■ 


9 


(B.8) 


where  5~-.  denotea  a  pseudo- inverse  of  r  .  as  defined,  in  the  previous  section. 
Indeed,  if  K  satisfies  (b.7-8)*  (3.6)  follows  at  once. 

Let  be  a  nonslnsular  pseudo- Inverse  of  Jqi*  (  Ouch  5  always 

exists;  it  can  be  found,  for  instance,  by  neans  of  relntionB  (a.1-2).')  The 
matrix 


'■^4l-^l^  ■  -  53)’  )  4.,  -  £2i4i%2  ”  -^1.2 


is  clea'*ly  nonnegative  definite;  hence  we  can  write 


■522  "  ^12ni'5l2 


We  must  show  that 


(B.9) 


(B.10) 


The  coi>aitji.cy»3  ffcnrarlaxice  catrlx  of  given  la  imlependept 

of  and  la  givoxx  hy  2^^ 


Finally,  ve  point  out  a  uflofui  fact! 

(B,14)  Xj^,  Xg,  Xj  are  gauBBiHc  randcm  vectors  and  the  pair  Xg,x^ 

l3  Independent,  then 
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fo,  prove  this,  let 


M  wmi 

h 


Then 

®QeiIv  =3^  "*  *feJ3b^^ 


-  tti  +  -  l&O 


(b.15) 


But 


fii,  - 4,1  hk 


42  2 


by  AssvoiQitl on.  0ubstltutliig  into  (b.X^),  ptroves  (b.1^). 

y«  hsve  alfwMj  resifiribed  In  ii|qpe&dix  A  thet  tb<p  ^•mmu-hvrcrttc  to 
nscetfiftrlly  to  the  Ixxrerse  If  the  letter  eaclsts.  But  if  the  corexience 

■etrix  of  the  conditioning  nuoAosi  vearlebliie  ie  alagtiAr,  the  peeudo-irrrersc  of 
this  aatrlx  (and  therefore  the  conUtlooel  expectation)  vlU  not  be  unique. 

This  is  only  a  silnor  ccepllcatlon.  ?0>r  Instance,  let  y^  and  y,^  be  tvo 
(scalar)  conditioning  Tartebles  and  essuee  that  y^  -  y^  (vith  parobfibility  1). 
Then  the  condltlogoal  ezpectetlan  of  ^  sger,  x,  givea  y.  as.!  y.^  =ay  be  vrittes 


yg)  - 


But  Bince  y^  ■>  ire  can  also  vrlte 

In  both  cases,  the  condltloosA  eiqRMiitsiAien  is  the  saMS  randoi  veriahlC:  althcatgh 
expik^aod  dlffarevtlyi  the  only  diffBuftswi  I*  that  la  the  flmt  saae 
of  the  coafflcleixt  aatrlx  la  1^,  la  tlm  aarrcai’  eaee,  it  la 


r^rlcal  cc^^tatlon*  1.  is  of  Intex^nt  to  the  nox*  of  tbe 

r  in  (B-e)  M  swfcU  as  pc«*iDl©.  In  uOis  ewe,  „  ^ 

Mtri.  —  _  ^  Tuan-rniiiri  vhich  hluB  the  oatfillest  Hv^^* 

Tl.  .  c.rt.1.  .p.cin<=  •«»•)  8.e  (A.7). 
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Fia  8  EXAMPLE  IK  20) 


T!^^  (SEC) 

Fia  10  SOLUTION  Of  THE  VARIANCE  tCrtJATiOH  CASE  A.  EXAMPLE  (R 50) 
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TIME  (SEC) 

Fia  11  SOUiTION  OF  THE  VAR  lANCE  EQUATIOH  CASE  B,  EXAMPlfTK  50) 


na  12  OPTIMAL  GAINS,  CASE  A.  EXWLE  (14.50) 
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Fia  13  OPTIMAL  GAINS,  CASES,  EXAMPlfUCSO) 


fl<i  M  (MiT  Slip  RESPONSE  OF  OPTIMAL  FILTER.  CASE  A.  IXAMPlf  114  50) 
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E  DF  OPTIII«AL  FILTER,  CrtSE  B, 


FIfi.  18  UNiT  STEP  RESPONSE  OF  OPTIMAL  FILTE?,  C\Si  B. 
EXAMPLE  (11 150) 
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fid  19  UNIT  STEP  RESPONSE  Of  OPTimiFILTlER,  CASES,  UvWli(U50) 
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